Building the Right Environment to Support AI, Machine Learning and Deep Learning
Environment: VC++ 6.0/7.0, Windows 98 and later, GDI+.
Sharpening and Blurring Bitmaps
In another article, I presented the use of color matrices to enhance (or deteriorate) bitmap pictures. With my demonstration project Tinter, you can adjust the tone (contrast, brightness, saturation, hue shift, and gamma) of bitmaps. Now, here is an updated and extended Tinter. The new version also lets you sharpen or blur a picture, and, as an extra, modify its tint.
There are lots of variations. For this project, we use the so-called discrete Gaussian convolution. It is generally regarded the best for image enhancement. For those interested in the historical context of such things, the name Gaussian honors Karl Friedrich Gauss (1777-1855), the great mathematician.
If you apply such a discrete Gaussian convolution to a bitmap picture, it gets blurred. Applications such as Adobe Photoshop, Corel Photopaint, or Jasc Paintshop Pro call it 'Gaussian blur.' It seems strange to blur a picture, and call it an 'enhancement,' but sometimes a picture really gets better by blurring it, for instance to hide noise, 'jaggies,' or other artifacts.
But blurring a picture is far more important as part of another operation. One such an operation is called 'Unsharp Mask.' Despite its somewhat confusing name, Unsharp Mask really sharpens an image (or, more correctly, seems to sharpen an image). There are other ways to sharpen a picture, but Unsharp Mask is by far the most common, and preferred.
Unsharp Mask works by partially subtracting a blurred bitmap from the unmodified original. You might say that some blurriness is removed to gain sharpness. It sounds like pure magic, but it really works, and the theory behind it is sound.
I wrote a class, QGaussFilter, that implements Gaussian Blur and Unsharp Mask. Although it has quite a complicated job to perform, employing the class is simple. Just create the class and call one of the following functions:
Bitmap * GetUnsharpMask(Bitmap * pSrc, REAL radius, REAL depth); Bitmap * GetBlur(Bitmap * pSrc, REAL radius);
The functions return a pointer to a newly created bitmap, containing a sharpened, or blurred copy of the bitmap pointed to by pSrc. The source bitmap is not altered in any way. The caller of the functions gets the ownership of the returned bitmap and is responsible for its destruction.
The radius parameter determines the number of surrounding pixels that are involved in the convolution, and therefore the intensity of the effect. For the mathematically inclined, it is identical to the standard deviation of the Gaussian distribution in the filter mask. It can be anywhere between zero and 8.0. For a more or less 'normal' effect to enhance photographic material, a value of 0.8 to 1.5 will suffice.
GetUnsharpMask() has a second parameter, depth. It determines the amount of blur that is subtracted from the original. Sensible values are between zero and 4.0 (it can be bigger, but it doesn't make much difference). Values of 0.7 to 1.0 are normal.
Convolution is a very costly operation, the more so for high values of radius. If radius is 8.0, the processor has to perform over two hundred multiplications for each pixel. As the number of pixels in a picture soon runs in the hundreds of thousands, we deal with many millions of multiplications. To avoid freezing the user interface, such intensive calculations can best be delegated to a separate user thread. QGaussFilter has two member functions that do just that:
void MakeUnsharpMask(Bitmap * pSource, REAL radius, REAL depth, CWnd * pMsgWnd); void MakeBlur(Bitmap * pSource, REAL radius, CWnd * pMsgWnd);
These functions do the same as both Get...() counterparts, but return immediately and let the calculations run in the background, in a worker thread. When the resulting bitmap is completed, the window pointed to by pMsgWnd gets notified by the special windows message QM_GAUSSFILTER (defined in QGaussFilter.h). The LPARAM parameter contains a pointer to the completed bitmap (WPARAM is not used).
If, for some reason, you want to interrupt the process, just call the function:
I cheated a bit in presenting the Get...() and Make...() methods. In fact, they all have a few more parameters, as you can see in the source files. To summarize:
- pRect points to a rectangle and lets you define a rectangular part of the bitmap to filter; if NULL (the default value), the whole bitmap is filtered.
- flags let you determine which of the three color planes (red, green, or blue—but read on) take part in the process, the default value meaning: all three planes.
- message defines the message sent to pMsgWnd; the default is QM_GAUSSFILTER (Make...() functions only).
The luminance issue
It appears that sharpening or blurring also works when it is only applied to the luminance of the picture, the black-and-white part. Not only does it work, in most cases it even works better. By modifying only the luminance, the colors don't get mixed up in the blurring process, which sometimes yields weird, new colors. There is also an algorithmic benefit: by convoluting only the luminance, the operation has to be applied to only one of the three color planes, so that the process (in theory) takes only a third of the time. Considering the huge amount of multiplications, this is a potential optimization not to be missed.
To make this work, we have to convert the RGB picture in such a way that the luminance is in one color plane, and the color information is in the two other planes. There are several ways to do this, such as converting to the HSL color space (hue, saturation, luminance). I chose the (slightly adapted) YUV color space, mainly used in television technique. Here, the Y plane contains the luminance, U contains the difference between blue and luminance, and V the difference between red and luminance. The advantage is that converting from RGB to YUV, and back, can easily be performed using a GDI+ ColorMatrix. The HSL color space is somewhat problematic in this respect.
So, what we do is: we convert the bitmap from RGB to YUV color space, next sharpen or blur only the Y plane, and finally convert the result back to RGB. It sounds like a lot of work and a lot of trouble, but actually it works great, and is relatively fast.
In the demonstration project, there is an option to switch 'Luminance only' on or off. Notice that for low values of radius and depth, the visual difference is almost indiscernible. Only for large values the results differ slightly, mainly around isolated, brightly colored spots in the picture. It is really a matter of taste which option you prefer, but one thing is for sure: 'Luminance only' makes sharpening and blurring more than two times faster.
The demonstration project, Tinter, extends on the first version, described here. Controls were added to select Unsharp Mask or Blur, and sliders to adjust radius and depth as well. For the (almost) 'real time' adapting of radius and depth I used the Make...() functions to calculate the convolution in the background. Internally, Tinter 1.1 works in the YUV color space exclusively. As soon as a bitmap image is loaded, it is converted to YUV. Only for display (and to save or print) it is converted back to RGB.
Extra: tint control
|Tint||hue = -144||hue = -72||hue = 0||hue = +72||hue = +144|
|amount = 0.2|
|amount = 0.5|
While I was at it, I also added controls to change the tint of the picture—something that, of course, should have been in the first version of Tinter to merit its name. The amount of tint can be set, and the hue. If the amount is positive, colors of a certain hue (say greenish) become stronger; if negative, they are washed out from the picture. Changing the tint of a picture is often the preferred way to correct its colors. To implement tint control, I extended my QColorMatrix class with one method:
Status SetTint(REAL phi, REAL amount);
Here, phi is the hue, measured as an angle between -180 and 180 degrees. An angle of zero degrees corresponds with blue. The parameter amount should have a value between -1.0 and 1.0, zero being neutral. For correction of color faults, a small value usually suffices. Tint is also useful to give monochrome pictures a subtle colorization.
I added some more goodies. It is now possible to toggle between the modified picture and the original one by pressing Tab—very convenient, if I may say so. Also, there are buttons to flip the picture, or rotate it at right angles.
On the Internet, there are vast amounts of information on subjects such as convolution, sharpening, and blurring. Many of it is very theoretical or specialised. A few starting points are:
- Convolution by David Young
- Filters defined using convolution masks by Wilfrede Orozco
And, of course, there is always the bible of computer graphics: Foley and Van Dam. (That's only the household name. In full you should say: Computer Graphics, Principles and Practice, 2nd Edition, by James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes. Addison-Wesley, 1995. ISBN: 0201848406. Rather old and not exactly cheap, but very good.)
Note: your system must support GDI+, which currently only XP does natively. However, other Windows versions can be upgraded. Also, VC++ 6.0 comes without the GDI+ headers. You may obtain them by downloading the Windows Platform SDK. The GDI+ headers are included with VC++ 7.0.
DownloadDownload demo project and source - 80 Kb