Home page» Image Processing» Managed Images

Managed Images and Performance

...or, how to destroy your application's performance

In the old days of Java, there was the ImageProducer/Consumer API which was good if you wanted to write a Web browser, but hopeless for image processing because there was no easy way to modify the pixels of an image. When Java2D came along, we got BufferedImage which solved all this and provided direct access to the pixels of an image. Now we can twiddle an image's pixels and draw complex things into an image using Graphics2D and everything is now wonderful and rosy. Unfortunately not. There are some severe problems associated with the internal implementation of BufferedImage which make using them very tricky if you want to get the best performance. The problem lies with managed images, and this note attempts to put into writing what I've found out about them to save other people from going through the same pain.

You see, internally, Java tries to use the graphics card in the computer to accelerate the rendering of BufferedImages. To do this, it keeps a mirrored copy of the image's pixels in the graphics card's VRAM and tries to keep the two versions of the image in sync. The problem arises if you try to tweak the image's pixels behind Java's back by getting hold of a handle to the array of pixels. Because Java2D doesn't know what you're doing with this array, it just gives up and decides not to accelerate that image any more. This makes drawing to that image in the future go slower, and I'm not talking just a little bit slower here, I'm talking factors of 50 or 100. I'm talking about 256x256 images which take half a second to draw. I'm talking rendering times which allow you to make a cup of coffee and read the newspaper while you wait for your polygon to draw. This problem is most apparent on Mac OS X. The Windows implementation of Java apparently does some very clever stuff behind the scenes to try and avoid it. Once your image becomes unaccelerated, there's no way to accelerate it again - you just have to create a new image and draw the old one into it.

The problem won't bite you unless you're accessing the pixels of an image in order to do some image processing, and it won't bite unless you're also drawing with Graphics. It's the combination of the two which really cause it, but you may find other slowdowns if you access an image's pixels. The methods which causes an image to become unaccelerated are getDataBuffer() and getSubImage(). If you call this method, your image is forever after doomed to work like treacle. Avoiding getDataBuffer is not sufficient as it's called internally by other methods. These are the things I've found which trigger the problem. I'm sure there are others:

• Raster.getDataBuffer()
• BufferedImage.getRGB()/setRGB()
• BufferedImage.getSubImage()
• Attaching an ImageObserver to a BufferedImage
• Filtering a BufferedImage with FilteredImageSource

So, how on earth are we supposed to do any image-processing if we can't access the pixels of an image? Well, firstly, you can access the pixels to your heart's content if you don't want to draw into it later. Secondly, you can do your processing, create a new image, draw the old one into it and throw the old one away. Thirdly, you can call the methods in Raster/WritableRaster which let you access the DataBuffer's data. This is really the only way to go, but it's a bit of a pain. Raster.getDataElements() and Raster.getPixels() are your friends. They let you access the pixels of an image without unaccelerating it. If you know that you are dealing with TYPE_INT_ARGB or TYPE_INT_RGB BufferedImages. then Raster.getDataElements() is pretty well equivalent to BufferedImage.getRGB(). Of course, this is less than ideal - it would be much better to have a way to access an image's pixels without having to copy them, but it's better than nothing.

One last thing. If you're going to mess with an image's pixels and draw into it. Whatever you do, don't do both at once or Java2D will get really confused. When you modify an image's pixels with, say setRGB, you're changing the version of the image in memory. When you draw with Graphics, you may be modifying the one in VRAM. Java2D has a really hard time keeping the two in sync, and you'll find that you can lose some of your drawing. Specifically, there are two things to worry about. Firstly, always make sure that you call dispose() on a Graphics object that you've finished with. If you don't you may find that the next call to setRGB will wipe out any drawing you did with the Graphics. Secondly, don't call setRGB (or friends) when you have a Graphics object active on the image. This can cause the drawing you did with setRGB to be wiped out.

While we're on the subject of performance of images. Watch out for ImageIO. The images returned by ImageIO are often in custom formats which can draw really, really slowly. For best performance, it's often best to draw any image returned by ImageIO into a new image of the appropriate pixel format for your system.