Demosaicing: Fundamental to RAW conversion

When you trigger your camera’s shutter, all the light that strikes the sensor is recorded as RAW data, but this needs to be converted into a different format before it can be displayed on a screen. This conversion can happen inside the camera, creating output files like JPEGs or TIFFs on the memory card, or the camera can be set to record exposures as RAW files for conversion later on using software like DxO PhotoLab or DxO PureRAW.

Whatever the process, for photographers who want to reproduce the highest quality and most lifelike rendering of a scene, the standard of this RAW conversion process can be crucial.

Demosaicing is one of the essential steps in this RAW conversion. It is an objective process that is fundamental to the faithful rendering of the recorded image and should deliver the optimum basis for any subsequent, more subjective, photo editing. In this article, we’ll explain what demosaicing entails as well as reveal how the approach used by DxO can be superior to the methods employed by other software.

What is demosaicing?

When you expose your camera’s sensor to light and look at the resulting image on the camera’s screen, or your computer monitor, it’s easy to forget that the photo is not recorded in full color. In fact, the sensor records a series of values that indicate the intensity of light at each of its photosites.

These photosites are intrinsically sensitive to all light without any ability to perceive individual colors, so manufacturers overlay color filters on top of the sensor. As such, only intensity data for red, green, or blue light is recorded at each photosite — similar to the cones in the human eye. In most camera sensors, these photosites are laid out in an alternating pattern of two green photosites for each pair of red and blue ones, so the distribution is half green, one-quarter red, and one-quarter blue.

How does it work?

Because only one-third of the actual color information in the scene has been observed, the rest needs to be extrapolated by an algorithm. There are many different demosaicing algorithms, but they all share the same objective – to take the recorded data and develop something that’s plausible to our eyes.

In its most basic form, for each pixel, those around it are sampled to predict the color. If one pixel shows a high level of green light and those around it have very low intensities of blue and red, it can be assumed that the correct color is a strong, though not pure, shade of green. But this simple algorithm assumes that each pixel has the same color as its neighbors, and while that might be true for the majority of pixels, it can be very wrong in many instances, for example where there is a sudden color variation, such as at an edge or on a texture.

To reach plausible results, the algorithm needs to make assumptions about the underlying scene. But these assumptions have to come from somewhere, so it needs to be trained.