Hey stackoverflow community!
I'm having an issue where a highly involved algorithmic program is using TOO LITTLE cpu utilization: somewhere between 3 and 4%. It is taking very long to return results, and I believe it's just not working hard enough.
Do any of you geniuses have any ideas why this would occur - if anything I would expect 100% utilization. One additional detail is that the program makes inserts into a sqlite3 database, and thus yes, there are a lot of JNI calls via the sqlite3jdbc library I believe. (Note that I wanted to defer these inserts with a PreparedQuery batch earlier, but this caused major memory problems - there's a lot of data).
Thanks in advance
UPDATE: Fixed. Yeah, I was just being a doofus, but I didn't expect that sqlite would start a new transaction and do so much overhead.
I now use a PreparedStatement and queue 32768 entries before insert - seemed like a good enough number to me.
Obviously the database calls are causing delays. Isn't it an option to create smaller batches and test if that helps?? Maybe you could parallelize the algorithm as well to have a queue somewhere taking results and another thread cleaning out that queue?
edit:
There are also some other problem areas:
All these factors should be taken into account