REGARDING THE USE OF THE BIOLOGICALLY INSPIRED RETINEX ALGORITHM IN ART CONSERVATION-RESTORATION

This paper addresses the problem of separating the illumination from the reflectance in images and compensating for non-uniform lighting. Recorded images of mural paintings suffer from significant losses in visual quality, compared to the direct eye observation when there are spatial or spectral variations in illumination. The visibility of detail in shadows is poor for recorded images and therefore an automatic computation is needed to improve them for better optical investigation. Images of medieval mural paintings, taken under very poor illumination conditions, are processed by the biologically inspired Retinex algorithm and by the equalize algorithm and the results are compared.


INTRODUCTION
The present paper represents an interdisciplinary approach of artworks investigation for the purpose of restoration and conservation, with respect to mural painting.
There has been an increasing interest in last decades in interdisciplinary approaches of all areas of artworks restoration and conservation.The motivation for this interdisciplinary interest lies not only in the historical and cultural significance of mural painting, but also in a number of special factors, specific to mural painting: their particular vulnerability, the limitations on controlling potential agents of deterioration, and the scale and expense of intervention, study and monitoring.
The conservation and restoration of mural paintings requires special methodologies and treatments that are directly related to their specific physical nature [1].Mural painting can be characterized by the following conditions: EVA 2009 London Conference ~ 6-8 July Oana Gui et al. _____________________________________________________________________  Their constituent materials have an open porosity, resulting in an easy accessibility of both liquids and gases, such as salt solutions, atmospheric pollutants, water vapour, solutions of materials used for conservation, etc.  They remain after restoration and conservation part of an open physical system, in contact with contiguous structures (walls, ground, roofs) involved in a series of physical and chemical dynamical events  The surrounding microclimate cannot be fully controlled in most cases.
Because of these conditions the intervention strategy must include, besides the appropriate conservation methodologies, the need for a periodic surveillance following restoration.
This periodic surveillance and the associated documentation imply a constant need for images, taken most often in very poor lighting conditions.Thus, an algorithm for image processing that compensates for insufficient and non-uniform lighting could be a very helpful tool for the modern art conservation-restoration domain.This paper draws a comparison between two image processing algorithms, one implemented by the Equalize command in Photoshop, and another one, Retinex, inspired by the human visual system.
The present work at the Restoration-Conservation Laboratory of the Art and Design University Cluj, Romania, intends to establish a proper methodology and a high quality standard in art conservation and restoration.

WHY BIOLOGICAL INSPIRATION?
We are witnessing now the emergence of a new philosophy in the engineering field, the Biomimetic or Biomorphing Engineering.Basically it's the concept of taking ideas from nature and implementing them in another technology such as engineering, design, computing etc.The engineering described by these terms encompasses all kinds of levels, from the molecular level to that of an ecosystem.The concepts are extremely relevant, given the fact that whole biological systems often outperform artificial systems.
One believes that studying and understanding the diversity of biological vision could give engineers and designers important yet simple vision cues if they accept to learn from nature [2], enabling them to re-engineer systems according to this new knowledge.One should try to discover some design ideas that can be transferred to artificial systems, and not simply copy nature [3]; that's why the use of 'biologically motivated' expression is preferred instead of 'biomimetical approach'.Biological organisms perform routinely a wide area of tasks that either cannot be replicated at all by artificial means (software, electronics or mechanical systems), or can be replicated only with much inferior performance and efficiency.Such tasks include obvious ones like intelligence, abstract thinking, pattern recognition in human beings, but also other tasks, more trivial and seemingly of lower level, routinely performed by creatures considered much less intelligent.Hence, an important idea is to study how biology achieves these tasks, in order to learn from them and be able to re-engineer systems according to this new knowledge.

Human perception
Receptors in the human retina respond to a range of light that is 10 billion to 1 in radiance.Yet, the optic nerve has only a dynamic range less than 100.This is possible only if the optic neurons transmit information about spatial comparisons [4].Observers easily discriminate details in scenes with dynamic ranges of 10,000 to 1. Outdoor scenes typically have dynamic ranges of 1,000 to 1 in radiances.Scenes with specular reflections, which contain reflected images of the sun, have much greater ranges.Print paper in actual viewing conditions has a dynamic range of less than 100 to 1. Tonescale transforms, such as S-shaped H&D curves, cannot render output images to match human sensations, compressing the highlights and shadows too much.But the equivalent of scene-dependent spatial-frequency filters could be automatically generated by means of spatial-comparison algorithms.
There are a few misconceptions regarding the way we actually 'see' the world around us, one of the most known being that there is a perfect resemblance between a photo camera and human eyes, the sensor of the camera being perceived as having the same functional role as the human retina.In reality, the sensor could be seen as a photon counter that measures values of intensity, whilst anatomic evidence, such as the centresurround organisation of the ganglion cells, suggests that ganglions in retina signal differences in brightness between adjacent parts of an image.
Humans, as well as other animals, must be able to recognize objects for what they are, under a wide range of lighting conditions, not for how much light they reflect or absorb.It seems the human visual system instead of registering the absolute light level removes it and uses another quantity to process the visual information, namely contrast.Contrast (C) measures the extent to which a part of an image differs from another.Given two surfaces with absolute luminances L 1 and L 2 , the contrast is defined as: The importance of this definition is that it is a property solely of the object we are looking at and does not depend of the lighting conditions.The range of values for contrast is between 0, when the two surfaces have the same luminance, and 1, when a surface is completely dark.

Colour constancy
Another common misconception is that the colour of an object in the visual field is fully explained by the wavelength of the light reflected from it.In reality, the visual system assigns colours according to the relative balance of short, medium, and long wavelengths across the whole visual field, so that light of a particular wavelength has no inherent colour in itself but may be perceived as any colour depending on the range of wavelengths in the surrounding visual field [4].
The image recorded by a camera depends on the physical content of the scene, the illumination incident on the scene, and on the characteristics of the camera.This leads EVA 2009 London Conference ~ 6-8 July Oana Gui et al. _____________________________________________________________________ to a problem for many applications where the main interest is in the physical content of the scene, such as art restoration and conservation.Consider, for example, the case of a computer vision application which identifies pigments by colour.If the colours of the pigments in a database are specified for tungsten illumination, then pigment recognition can fail when the system is used under the illumination of blue sky.This is because the change in the illumination affects pigment colours beyond the tolerance required for reasonable pigment recognition.
Thus the illumination must be controlled, determined, or otherwise taken into account.The ability of a vision system to diminish or, in the ideal case, remove the effect of the illumination and therefore 'see' the physical scene more precisely, is called 'colour constancy' and is characteristic of the human vision system.
One consequence of our own colour constancy processing is that we are less aware of colour constancy problems which face machine vision systems.These problems become more obvious when dealing with image reproduction, which is an important part of every restoration-conservation project.For example, if one uses indoor film (balanced for tungsten illumination) for outdoor photography, one will get a poor result.The colour change is much larger than expected based on human colour perception.
Many visual phenomena can be modeled by spatial comparisons.Colour constancy, visibility of gradients and edges, appearance of transparency, and colour gamut transformations are closely related; they share a common property, namely they can be explained by human spatial-comparison mechanisms.

DIGITAL IMAGING
This leads us to the need for a model that matches relationship between colour constancy and image reproduction.Illumination modeling is also beneficial for image reproduction and image enhancement.In the above example, taking a good picture required selecting the film based on the illumination.However, choosing among a limited number of film types provides only a rough solution, and has the obvious limitation that human intervention is required.Digital image processing has opportunities for improved accuracy and automation, and as digital imaging becomes more prevalent the demand for image manipulation methods also increase.Often modeling the scene illumination is a necessary first step for further image enhancements, as well as being important for standard image reproduction [5].
Digital imaging often uses the terms 'linearity' and 'gamma' to describe the behaviour of devices such as displays, scanners or digital cameras.Scanners, cameras, printers, and even colour spaces have 'Tone Response Curves'.Tone Response is simply a group of numbers that describes the relationship between input and output values over the entire range of the device.How these curves are plotted is somewhat important to how we perceive colours.Many people use the term 'gamma' to describe these curves.Most often the term gamma is used incorrectly.Gamma in colour science represents the argument of a power-law mathematical formula, used to determine the output from the input of a device gamma input output  . Various values for gamma produce different curves, so a single number is used to describe this type of curve.

EVA 2009 London Conference ~ 6-8 July
Oana Gui et al. _____________________________________________________________________ In colour management, the display gamma is often defined as a calibration target value.If the input and output values were linear, a straight-line curve could be plotted, and a numeric value based on the gamma formula could be assigned.With this formula, a flat linear gamma curve has a value of 1.0 thus preserving the initial values of the channels.The values increase as the curve becomes steeper.Most displays have an input-output relationship that follows a simple gamma curve of 2.0 to 2.2.On a typical standard 2.2-gamma CRT display, an input intensity RGB value of (127, 127, 127) only outputs about 22% of its possible brightness range per pixel rather than the expected 50%.To obtain the correct response, a gamma correction is used in encoding the image data, as explained above.Displays are among the few devices that exhibit input-output behavior based on the gamma formula.Technically, it's incorrect to use a gamma value to define other devices.The input and output values of a scanner or printer produce complex curves, which are better described as 'tone response curves' when discussing the devices' input-output relationship.In the study of the perception of colour, one of the first mathematically defined colour spaces was the CIE 1931 XYZ colour space, created by the International Commission on Illumination (CIE) in 1931.
A few processes do behave in a linear fashion, such as the sensor on a digital camera.However, the human visual system is not linear.If our visual system was linear, emerging from the dark into full sunlight would be a painful experience.In fact, many human sensations are non-linear, which protects us from serious sensory overload.
The sensor of a digital camera could be seen as a photon counter that makes a linear record of the light striking it.Each level recorded produces an equal level of corresponding data in the RAW file.If one photon hits the sensor, representing the first dark tone that can be recorded the result is level one in the digital file.If 2.048 photons hit the sensor, representing the brightest value it can record, the result is level 2.048 in the digital file.In an image with 2.048 tones and six stops of tonal data from black to white, the first stop of highlight values contains 1024 steps, which represents half available data, while the last stop has only 32 bits of data.Without proper exposure, that last stop of shadow detail will contain more noise than readable data.
A nonlinear tone response curve may or may not follow the gamma formula.In analogue photography, it is common to see the effect of nonlinearity in film, paper and similar light-sensitive materials (the S-shaped Hurter-Driffield curves, also known as the 'characteristic curve').When light strikes film, the resulting density is not linear.Film's response is less sensitive in shadows and more sensitive in highlights with a linear response between both areas of these two curve points.Digital images can have a linear or nonlinear tone curve as well.RAW format images have a linear encoded gamma of 1.0.If viewed on a display that does not compensate for this with an embedded profile, the images look dark.Some RAW converters allow images to be rendered in this encoded linear colour space.When converting the files to an RGB working space, which has a specified tone response curve, the histogram will be remapped, too.
Images, colour spaces and devices all have tone response curves.Some are simple and follow the gamma formula, others are not.That is why we propose the use of RAW files and Retinex algorithms that match human visual system for restoration-EVA 2009 London Conference ~ 6-8 July Oana Gui et al. _____________________________________________________________________ conservation of fresco, as a non-destructive technique of image acquisition and not the gamma curves used in Photoshop.In Photoshop, gamma relationship described by the Curves dialog represents the input and output values of pixels; altering the relationship between adjacent points alters tone or colour.

THE RETINEX MODEL
Recorded colour images of artworks, especially large ones like fresco, suffer from significant losses in visual quality, compared to the direct observation of the scene by the eye, when there are either spatial or spectral variations in illumination.The visibility of colour and detail in shadows is quite poor for recorded images and therefore a general purpose automatic computation is needed to improve images and allow for a better optical investigation of fresco in art restoration.Retinex algorithms share a common mechanism with vision: they are based on spatial comparison of pixels in all regions of the captured image and use multi-resolution spatial comparison techniques to render high-dynamic range scenes.These algorithms, now available also in commercial products, mimic human image processing.
The Retinex algorithm developed by Land and McCann [6] is one of the most famous attempts to model and explain how the human visual system (HVS) perceives colours.Land and McCann performed several experiments and demonstrated that colour perception is not a trivial sampling of light distribution.Our visual system automatically performs a 'computation' on the acquired signal that makes our perception more complex than a simple signal acquisition.To underline this they called this computation Retinex, term derived from 'retina' and 'cortex'.
The basic Retinex model is based on the assumption that the HVS operates with three retino-cortical systems, each processing independently the low, middle, and high frequencies of the visible electromagnetic spectrum.Every independent process forms a separate image that determines a quantity L, called lightness.When Retinex is applied on digital RGB images the triplet {LR, LG, LB} of lightness values in the three chromatic channels is the information that determines, by superposition, the perception of what we call the colour of every pixel of the image.

The Retinex algorithm
This implementation of the Retinex algorithm was written in C# 3.0 and runs on Microsoft's Windows, being able to process multiple formats of images like TIFF, PNG, and JPEG.When Retinex is applied to coloured images, each RGB channel is processed independently to determine the lightness for each pixel by locally inspecting all nearby pixels that influence the reference pixel i.The formula used to compute the lightness value L of a pixel i, positioned at (x,y) in the photo, in each of the RGB channels is [3].
EVA 2009 London Conference ~ 6-8 July Oana Gui et al. _____________________________________________________________________ A certain number of paths have been considered and the average of the norms was taken as the lightness for each pixel.Each of the paths starts at a random pixel j from the neighbourhood of pixels surrounding pixel i; the neighbourhood was considered to be a rectangle of a certain width and height and the entire random path was constrained to remain within the bounds of the rectangle and margins of the photo at all time.
Given the path γk, and , then the sequence of pixels has to meet , where the only possible moves from were allowed to be only diagonally.
The norm of the path k j i l , uses the ratio between the intensities of each pair of consecutive pixels . When using photos of 8-bit colour depth pixels, spans its intensity over the entire interval ] 255 ... 0 [ ; for technical reasons, the first value that belongs to the interval was removed to avoid division by zero, or the logarithmic scale yielding infinity; furthermore, the intensities were normalized taking into consideration the removal of the first value.
As seen in the above formula, Retinex has a reset mechanism; if, during a path computation a lighter area is found, the cumulated relative reflectance is forced to zero, making the average computation restart from this area.The effect of the reset mechanism is to consider the lightest area of an image as the reference value of the colour white.

Post-Retinex processing
The post-Retinex processing consists of several stages that are needed to overcome the logarithm and the normalization that was done to the input values.Exponentiation of the Retinex output simply compensates for the logarithm which was applied to the input data.Scaling to white is required because our implementation of the Retinex algorithm normalizes each of the channels to 1.
During the past 40 years, the Retinex model has inspired a great variety of implementation and discussion [6,7,8].Provenzi et.al [6] have re-written the basic Retinex algorithm in mathematical language; their definition is completely independent EVA 2009 London Conference ~ 6-8 July Oana Gui et al. _____________________________________________________________________ of the peculiarities of the implementation, and it can be used as a common background for various Retinex versions.The Retinex algorithm depends on several parameters (such as threshold, number of paths, and iterations): Provenzi et al. [6] showed that the qualitative behavior of Retinex in relation to the variation of these parameters can be predicted by using the mathematical definition.The dark image was than processed first by Adobe Photoshop using the Equalize command.This command redistributes the brightness values of the pixels in an image so that they represent more evenly the entire range of brightness levels, by finding the brightest and darkest values in the composite image and remapping them so that the brightest value represents white and the darkest value represents black.Photoshop then attempts to equalize the brightness by distributing the intermediate pixel values evenly throughout the grayscale.
Next, figure 1(d), the image was processed by a Retinex based image processing algorithm, to improve its appearance.The algorithm was written only for research purposes, and thus has not been optimized according to time related criteria.The histograms of the four images were acquired by means of NI Vision Developing Module.The histograms represent the total number of pixels (vertical) that have the same tonal level with regard to the total number of 256 tonal levels (horizontal).To plot a RGB histogram the computer scans through each of the RGB brightness values and counts how many are at each level from 0 through 255.
The region where most of the brightness values are present is called the 'tonal range'.However, there is no one 'ideal histogram' which all images should try to mimic; histograms should merely be representative of the tonal range in the scene.

CONCLUSIONS
The image in figure 1 (a), taken under daylight luminance conditions, is an example which contains a very broad tonal range, spanning the entire mid tones region but also covering highlights and shadows.This translates into a histogram, figure 2(a), which has a good distribution of pixel count over the entire interval, yielding a good contrast between the light and dark areas in the scene.
The image in figure1(b), taken under dark luminance conditions, contains very few mid tones and plentiful of shadow regions.This translates into a histogram, figure 2(b), which has a high pixel count on the far left of the plot.Due to the condition under which it was taken there is not much contrast or texture to emphasize as the mid or light pixels were not recorded in the photo.
One solution to the above problem was to apply an equalization filter to the image.The equalization of the dark image, figure 1(c with more distinguishable textures that unfortunately has several problems.Clipping has occurred since one can readily see the shadows and highlights pushed to the edge of the chart.Although some clipping is usually acceptable in some regions, this leads to loss of texture which is unrecoverable.The second problem is the way the equalization filter works, in that it approximates some centroids of multiple values and associates them to their representative, yielding the irregular shape of the chart; to be noticed is the 'shark tooth' allure of the curves, figure 2(c), a sign that not all tone levels are truly represented and also a sign of a 'posterized' image.
The Retinex algorithm that was applied on the dark image, figure 1(d), compared to the equalized image is much smoother in colour changes and is able to create a resemblance of the image that was taken under daylight luminance conditions, figure 2(d).The Retinex algorithm was able to transform automatically the initial photo that contained only shadow tones to a photo that contains both shadow and midtones.As contrast is a measure of the difference in brightness between light and dark areas in a scene, this algorithm was able to recover texture from an image that looked almost flat.
Useful observations could be drawn by comparing the histograms of the four images.The comparison is indicating that the colour distribution of the image depicted in figure 1(b) is strongly altered when using the Equalize command.Although this image processing technique improves the original image, it should be avoided when colour distribution is involved in conservation-restoration techniques.
All these qualify the automated processing of darker images by means of Retinex algorithms as a more objective method for conservation-restoration field as compared to the Equalize command in Photoshop.

Figures 1 (
Figures1 (a) and (b) represent two jpeg images taken with a digital camera in extremely different illuminance conditions.Due to the damaging influence of light, the University of Art and Design is keeping during restoration the artworks (icons painted on wood, from the Museum of the Rohia Monastery) in an almost dark room and no flashes are allowed.Consequently, the images look extremely dark.The dark image was than processed first by Adobe Photoshop using the Equalize command.This command redistributes the brightness values of the pixels in an image so that they represent more evenly the entire range of brightness levels, by finding the brightest and darkest values in the composite image and remapping them so that the brightest value represents white and the darkest value represents black.Photoshop then attempts to equalize the brightness by distributing the intermediate pixel values evenly throughout the grayscale.Next, figure1(d), the image was processed by a Retinex based image processing algorithm, to improve its appearance.The algorithm was written only for research purposes, and thus has not been optimized according to time related criteria.The histograms of the four images were acquired by means of NI Vision Developing Module.The histograms represent the total number of pixels (vertical) that have the same tonal level with regard to the total number of 256 tonal levels (horizontal).
) is such an example, generated an image