I'm working with Java project that requires very advanced manipulations of images. In fact, I'm doing most of the manipulation using OpenCV, and I'm using JNI to wrap around the OpenCV functions that I need. I am extremely satisfied with the performance OpenCV gives, the people who wrote the OpenCV code deserve great great credit for the code. In sharp contrast to what I experience with the code Java devs wrote.
I started out optimistic over the choice of my programming language, my first working iteration of the project works fine, but its performance is nowhere near to realtime (getting about 1 frame per 2 seconds.) I've done some optimizations of MY code and its helped a lot. I've been able to push the frame rate up to about 10-20 frames per second, which is great, but what I'm finding is that to do any further optimizations I have to rewrite Java code to do the same thing but 10-20x more efficient.
I'm appalled at how the developers of Java pay very little attention to performance, especially when writing the classes for Media related classes. I've downloaded OpenJDK and I'm exploring the functions I'm using. For example, there is a function under the Raster class called getPixels(...) and it gets the pixels of the image. I was expecting this function to be a highly optimized function in the source code, with several calls to System.arrayCopy to further optimize performance. Instead what I found was extremely "Classy" code, where they are calling 5-6 different classes and 10-20 different methods just to accomplish what I can do in one line:
for (int i =0; i < n; i++) {
long p = rawFrame[i];
p = (p << 32) >>> 32;
byte red = (byte) ((p >> 16) & 0xff);
byte green = (byte) ((p >> 8) & 0xff);
byte blue = (byte) ((p) & 0xff);
byte val = (byte)(0.212671f * red + 0.715160f * green + 0.072169f * blue);
data[i] = val;
grayFrameData[i] = (val & 0x80) + (val & (0x7f));
}
The code above transforms an image to grayscale and gets the float pixel data, in roughly 1-10ms. If I wanted to do the same with Java built in functions, the conversion to grayscale itself takes 200-300ms and then grabbing the float pixels takes about 50-100ms. This is unacceptable for real time performance. Note to get a speedup, I make heavy use of bitwise operators, which Java devs shy away from.
I understand that they need to handle the general case, but even so, can't they at least give options for optimizations or at the very least a warning how slow this code may perform.
My question is, at this late point in the development (I already have my first iteration, not I'm working on a second that performs more in real time) should I bite the bullet and switch over to C/C++ where I can fine tune things a lot more, or should I stick with Java and hope things will become more realtime friendly so that I won't have to rewrite already implemented Java code to get a speedup.
I'm really beginning to become disgusted with how "classy" and slow Java really is. The amount of classes there are seems like overkill.
I've done computer vision work with Java, and I think it is perfectly usable for computer vision and realtime stuff, you just have to know how to use it.
If you need help optimizing your code, I'd be glad to assist -- for example, I can tell you that you will probably get a performance boost by making a method
`public static final int getGrayScale(final int pixelRGB){
return (0.212671f * ((pixelRGB >> 16) & 0xff) + 0.715160f * ((pixelRGB >> 8) & 0xff) + 0.072169f * ((pixelRGB) & 0xff));
}`
and using this in your for{pixels} loop. By using a method call, the JVM can much more heavily optimize this operation, and can probably optimize the for loop more too.
If you've got RAM to burn, you can create a static, final lookup table of output grayscale bytes for all possible 24-bit pixel pixel colors. This will be ~16 MB in RAM, but then you don't have to do any floating point arithmetic, just a single array access. This may be faster, depending on which JVM you are using, and whether or not it can optimize out array bounds checking.
I would strongly suggest that you take a look at the code for the ImageJ image processing app & its libraries, specifically ij.process.TypeConverter. Just like your code, it relies heavily on direct array operations with bit-twiddling and a minimum of extra array creation. The Java2D libraries (part of the standard JRE) and the Java Advanced Imaging(JAI) library provide other ways to do image processing directly on image data rapidly without having to roll your own operation every time. For Java2D, you just have to be careful which functions you use.
Most of the "class-iness" is due to supporting multiple color models and storage formats (I.E. HSB images, float-based color models, indexed color models). The indirection exists for a reason, and sometimes actually boosts performance -- the BufferedImage class (for example) hooks directly into graphics memory in recent VMs to make some operations MUCH faster. Indirection lets it mask this from the user a lot of the time.