[2015/12/22] Univ. Prof. Oliver Bimber and his team at the JKU Institute for Computer Graphics have had another successful breakthrough: the reconstruction of color images by using multi-layer plastic film.

Conventional image sensors measure red, green and blue components in color images on adjacent image elements (pixels) that contain different color filter. The spatial resolution found in the resulting color images are in fact only a third of the actual sensor resolution.

Not so with the newly developed sensor as the color components are measured in different layers to one another without affecting the image’s spatial resolution. This is enabled by the sensor film’s transparent properties. In this case, the measurement signal in each layer is similar to that in a computer tomography but the difference is that light is captured instead of x-rays.

The Program Can Gauge Color Components
Current research also shows that conventional images, such as pictures with natural image statistics, require fewer layers than color channels that the resulting image should actually contain. Conventional RGB images only need two layers, for example, that physically only measure the green and blue channels. The statistically given, most likely share of the red channel can then be gauged by using a machine learning approach (artificial intelligence is applied so the machine can use available information to learn independently). Project manager Prof. Bimber remarked: "In principle, the different color components had to be measured at adjacent points before but now we can measure two components on the same spot and calculate the third."

In order to explore the correlation between red, green and blue components, the team analyzed thousands of images from image databases such as Flickr. Sophisticated algorithms now allow the sensor to learn these correlations and apply the acquired information in order to reconstruct the missing color channel with known channels – and with surprisingly good quality.

Image: The reconstructed images come very close to looking like the original image.
Details about the new color sensor will appear in the renowned Journal of Optical Society of America (OSA), Optics Express. The project was financed by Microsoft.

[Christian Savoy, Translation: Nadine Lichtenberger]