My Java app reads in thousands of text files, goes through the lines and parse the text into floats and does some calculation with them then saves the results into some files, now I'm using multiple thread to execute them simultaneously, still need about an hour to run the whole app.
Here is a simplified pseudo version of what my app does :
String Lines[]=ReadFile("abc.txt"),Items[];
float its[];
for (int i=0;i<Lines.length;i++)
{
Items=Lines[i].split(",");
its=new float[Items.length];
for (int j=0;j<its.length;j++) its[j]=Float.parseFloat(Items[j]);
do some calculation with : its[]
}
Do more calculation, save results to a StringBuffer.
Save StringBuffer to ("abc_result.txt");
I've read that Rootbeer is able to shift some workload to GPU and speed up the process, so my questions are :
[1] Can Rootbeer handle text parsing and file I/O on the GPU ? Or do I need this portion done on CPU and only do the calculation on GPU ?
[2] Would my app benefit from Rootbeer ?
The answer, as so often is: it depends.
Rootbeer will be able to shift some work to the GPU. But there are lots of caveats.
Now, lots of the processing in your application, by the look of things, is to do with processing the strings. You're invoking .split()
on a String
, which breaks it up into a String[]
; then you go through the bits and read each one to convert to a float
; then you do your calculations. This is quite heavy work, especially on the garbage collector, which has to get rid of all these String
s.
I did some detailed analysis of something very similar, using integers instead of floating point, in this question, and it led to some detailed discussion. The conclusion there was that it made a huge difference to abandon the String
-based approach and read the files in character by character, turning them manually into integers. Almost certainly something very similar will help you with producing floating point numbers.
So, in short, Rootbeer might help, but you have a lot you can do before you try something that drastic.