Reflectance Transformation Imaging and ImageJ : Comparing Imaging Methodologies for Cultural Heritage Artefacts

The imaging of cultural heritage sites and artefacts is now a highly technical process with many tools and methodological approaches available to archaeologists, architects, museum curators and artefact conservators. Imaging studies at the Australian Institute of Archaeology (AIA) have been directed principally to the recording of the artefacts within the collection. Several imaging tools have been used, including optical microscopy, pseudo 3D photography using a translation rig, flatbed scanning and Reflectance Transformation Imaging (RTI). RTI is an important tool for the on-going Cuneiform in Australian and New Zealand collections (CANZ) project, one output of which will be a web-site from which researchers will be able to load the interactive RTI files that can be viewed using Cultural Heritage Imaging (CHI) algorithmic rendering tools. Where publication of the AIA artefacts through journal articles and monographs is to be undertaken, other imaging techniques are being investigated to capture or enhance detail in a single image.


INTRODUCTION
The invention of photography in the early 1800s provided a way of capturing the subject "as-is".Among its early applications were attempts to capture images of important historical sites, including archaeological sites, excavated artefacts and to record the archaeological dig itself.Photographic imaging was, and continues to be, an important tool in recording the past as it represented by artefacts at the time of imaging.
The application of imaging to the preservation of cultural heritage requires continual development of imaging techniques.
One important area is image processing where images are changed from their original capture to provide; 1. improvements in appearance of the image 2. enhancement of obscure features 3. extraction of elements from the image that may be used for quantitative measurements 4. appearance of texture and 3D shape Modern imaging technology has developed significantly over the past 50 years, the technology evolving through the application of algorithms based on mathematics, aided significantly through digital imaging.Many processing tools have been developed, e.g.filters, edge enhancement algorithms and reflectance models.Within many tools are combinations of mathematical algorithms that may have been developed initially for other purposes, see for example the work of Gautron et al. (2004) and the development of Hemispherical Harmonics for modelling reflectance.Image manipulation at the pixel level is now a multiplexing of technologies, the aim of which is to extract the maximum information from image sets.The field of cultural heritage imaging, however, still has to address multiple requirements including providing researchers with enhanced perception of detail, purveying the sense of shape and texture for visitors to museums and accurate rendering in technical publications and records, see discussions by Payne (2013) on imaging techniques and Arnold (2008) on the use of digital representation and associated metadata.
The Australian Institute of Archaeology (AIA) is using two significant imaging technologies for recording and promulgating detail on cultural heritage artefacts on an interactive web site.These are Reflectance Transformation Imaging (RTI) and ImageJ; two significantly different processing methodologies, but, as we will demonstrate, have synergies that can be exploited for specific imaging tasks.
RTI is principally being used for the Cuneiform in Australia and New Zealand collections project (CANZ), Saunders et al. (2016) and ImageJ is being used for image enhancement using sets of images of Roman coins and for an incised text on a stone block excavated in Kourion, Cyprus, Davey et al. (2014).
In this paper we describe experiments using images captured using the RTI methodology and processed using ImageJ.We compare the ImageJ results with the same set using polynomial texture mapping (PTM) and/or hemispherical harmonics (HSH) to model the reflectance functions at each pixel of the image.

IMAGING ARTEFACTS
In this work 20 MPixel digital single images (no cropping) were captured in camera RAW using the AIA illumination dome, described below.
The images were processed using: i) RTI modelling based on either PTM or HSH; ii) ImageJ Z-Project stacks.

Reflectance Transformation Imaging
RTI is "a digital (image) acquisition process that captures sets of images of a subject from a single view (point) under varying lighting conditions … reflectance functions are modelled from the captured data, making it possible for the user to interactively relight the subject", Palma et al. (2010).
The RTI methodology was developed initially by Malzbender et al. (2001) of Hewlett-Packard and later developed for the public domain by Cultural Heritage Imaging (CHI).Algorithmic rendering (AR) of the RTIViewer developed by CHI ( 2013) now provides a suite of tools (i.e.algorithms) to enhance the perception of texture, shape and shadowing on the surfaces of artefacts and structures.
Thus, while RTI provides a way of enhancing the impression of texture, etc, it is unfortunately at the expense of imaging detail (necessary for enlargement of areas and some post processing methodologies) because the processing currently uses JPEG files that are lossy image files.

ImageJ
Following its release in 1997 (initially as NIH-Image), ImageJ has been progressively expanded over many years from the original software developed by W. Rasband under the sponsorship of the U.S. National Institutes of Health (NIH); see Schneider, Rasband and Eliceiri (2012) for a history of the development of ImageJ.While retaining its original core design intent, ImageJ now has extensive features that enable the editing, displaying, and analysis of individual images and image sets.Most importantly, it has the capabilities to process 8-bit, 16-bit and 32-bit reading most standard image formats, including TIFF, GIF, JPEG and DICOM.
ImageJ is an important imaging tool in the fields of biological and medical science, Abramoff et al. (2012).It has also been investigated for applications in archaeological research, Christens-Barry (2012).ImageJ has been used with focal stacking, Figueiredo et al. (2013), for shape/region counting Freeman et al. (2013) and the application of D-Stretch®, Le Quellec et al. (2015).
A method related to focal stacking is the use of ImageJ Z-Project to process images taken from a single camera position and with each image at a different lighting angle.

Artefacts Studied
In this work, we image two artefacts from the AIA collection, Figures 1 and 2.   Papyrus fragment IA15.554, Figure 2, is believed to be from the Coptic monastery at Bawit, Egypt and to date to the 7th Century.
The text is currently the subject of an AIA program studying a number of papyrus fragments in the collection.

Physical Set-up
All images were captured using the illuminating dome of the AIA.This consisted of a fully enclosed multiple-light dome with an equatorial dia. of 1metre, Figure 1.The illuminating dome and control software was developed by eAustralis Pty. Ltd.
The illuminating dome uses a single string of 35 white light emitting diodes, (LEDs), positioned around the dome and fired sequentially at the same time a camera is triggered.With appropriate camera settings, sufficient illumination time is allowed for each image capture.An SD card with 80mb/sec write time was sufficient to assure sequential image capture of all images.
The camera used for the imaging was a Canon EOS T4i/650D with Canon EF 50mm and 100 mm USM prime lenses.

Image processing methodologies
The workflow for image processing is shown in Figure 4.
Initially all 35 Canon RAW images were converted to 16 bit TIFF and JPEG images using Canon EOS Digital Software Version 25.2.In some cases, exposure and contrast adjustment was made to individual RAW images prior to image conversion.The RAW images were converted to JPEG files, for RTI processing and to TIFF files for ImageJ processing.
In this work flow DNG files can be used to provide images for archival storage.These were not used in this program of work.
The TIFF images for image processing through ImageJ were selected on the basis of their exposure quadrants within the RTI dome.The following quadrants were used, NW, NE, SW, SE.
This process, discussed below, resulted in 8 -12 images per quadrant for processing.

Imaging using ImageJ
The principal processing methodology was to use Z-Project, with minimal pre-processing of the TIFF images files prior to "stacking".
Z-Project combines many images into one (the Z direction projection) by processing the individual pixel data in each slice to create the final single image.Different projection methods are used to highlight specific pixel data within the stack.This methodology is used in many applications including the extraction of pixel data from MRI and X-ray tomographic slices.
Z-Project processing in ImageJ consists of AVG, MAX, MIN, SUM, STD and MED where, for example, AVG creates an image where each pixel of the resulting Z-Project "stores" the average of the intensity of all pixels contributing to the projection; STD is an image created from the standard deviation of the intensities of each pixel in the z direction and MED the median intensity over all pixels in the z direction (etc).Experimentation with the projection algorithms provides a range of enhancement outcomes such as those widely discussed in the journals in relation to medical imaging.STD and MED were found suitable for the current work.
In this work the "stacks" consist of images taken at the same camera (imaging) position, but with different lighting angles.If, as we show in Figure 5, all 35 lighting angles are included in the stack, the Z-Project presents an image with limited enhancement of detail.This imaging methodology is considered not to be appropriate for cultural heritage imaging as it produces a "Projection" with no specific shadowing and tends to lack detail achieved with, for example AR using a PTM of HSH rendering of the reflectance function, as shown below.
One of the tools within ImageJ is the introduction of shadowing [ process > shadows ] to enhance the perception of detail using image enhancement algorithms.Rather than introduce shadowing, in this work we selectively admit images with specific shadowing based on the quadrant of illumination, as shown in Figure 6.The resulting stacks use 8-12 images that have otherwise not been manipulated or enhanced -although some exposure correction was undertaken in processing RAW > TIF.The 8-12 images from an illumination quadrant were processed using [stacks > Z-Project].While a number of single image outputs using the Z-Project algorithms were generated we show STD and MED in this paper.
Further processing of the Z-Project image output was possible, for example, smooth, sharpen, enhance contrast, find edges and apply shadows.
From the perspective of a single image for publishing, we considered it unnecessary to apply these tools to the images produced in this work.These tools, however, may be applied for the quantitative study of the surface of the artefact.

Imaging IA5.074
The obverse face with the seals was algorithmically rendered using CHI RTI Viewer Ver.1.1.Figure 7 provides a set of images generated using the PTM fitter and using the default AR mode.In this example, the low angle lighting from the SW provides clear imaging of the text.However, because of the convex surface, limited detail over the whole tablet surface was achieved.
The ImageJ Z-Project using STD provides, in a single image, a detailed view of much of the text and accommodates the convex shape of the tablet.This methodology provides clear imaging of all the text on the obverse face.Detailed AR of the seals can be achieved using specular enhancement in the RTI Viewer as shown in Figure 8.An equivalent enhancement using ImageJ has yet to be investigated.

Imaging IA15.554, Papyrus Fragment
For the second experiment the papyrus fragment was imaged and processed using both RTI and ImageJ.As with the small cuneiform tablet the illumination dome was used to capture 32 images and in this experiment the 3rd order HSH fitter was used.The algorithmic rendering in Figure 9 used the default AR of the RTI Viewer.
The ImageJ Z-Project utilised eight (8) 16-bit TIFF images with MED processing for the projection.
The results are shown in Figure 9.
The "selective reveal" of RTI showed shadowing but as expected, to a much lesser degree than for the convex cuneiform tablet.In this case the HSH modelling generated from the higher altitude lights produced rendered images displaying significant over-all surface detail.As shown in Figure 9, there appears to be less advantage using ImageJ to achieve a uniformly illuminated, single image showing over-all detail of the artefact.It should be noted that ImageJ Z-Project with a specific quadrant lighting bias produces a single image that conveys a sense of texture while clearly showing all the text.

Comparing Imaging Methodologies
The results have shown that generation of images that provide an impression of texture and detail can be achieved using RTI and ImageJ.Other methodologies may also be appropriate but were not investigated in the present work.We note that the Cuneiform Digital Library Initiative, CDLI, uses flatbed scanning as the major imaging tool however RTI is also used by some of the institutions participating in the project.CDLI is a joint project of the University of California, Los Angeles, the University of Oxford, and the Max Planck Institute for the History of Science, Berlin, see http://cdli.ucla.edu/As noted above, the convex shape of the tablet makes imaging problematic, resulting in the "selective reveal" of RTI imaging.This effect is not so evident in the case of the papyrus fragment.
Where "selective reveal" is prominent, multiple images may be required in publications to convey the complete view of the textured surface, viz. of inscribed texts (etc) across a curved surface.
ImageJ Z-Project processing provides a computationally different way of producing an image, or series of images, of an artefact.ImageJ does not produce a mathematical representation of reflectance at every pixel captured to create the texture map, but instead uses pixel information, for positionally-similar pixels, to create a projected image from processed data.In this work the Z-Project lighting biases, viz. the NE, NW, SE and SW quadrants, are chosen by the user, in much the same way as those chosen in the RTI Viewer.This methodology uses a more general light biasing than RTI but has synergistic, biasing in that more lights are from lower angles.The resulting shadows are not from a point light source as in the case of RTI but are "averaged" over the image.
For some applications ImageJ processing currently has an advantage over RTI because it processes TIFF and other non-lossy file formats.For many illustration or publishing of artefacts it is arguable as to whether the additional image information is necessary, however quantitative studies of artefacts, such as particle counting, damage assessments (etc) may use the processing capabilities of ImageJ.As noted by Arnold (2008, p 161), "if [data] has not been recorded, analysis cannot follow".
From this experimental program we summarise the advantages and disadvantages of RTI and ImageJ methodologies used in the study of cultural heritage artefacts.

CONCLUSIONS
There are clear advantages and disadvantages of the two methodologies studied.Over-all, RTI presents the most efficient way of capturing (as texture maps) and re-processing images of cultural heritage artefacts.As we have shown, for convex, textured artefacts, the ImageJ Z-Project methodology produces a single image that reveals much of the surface detail unlike the "progressive" reveal of RTI.We consider that such an image would be suitable for publication whereas the RTI methodology provides for study of specific areas using controlled lighting directions.RTI conveys significant benefits to researchers who need to study texture and patterns (etc) on the surfaces of artefacts Clearly the two imaging methodologies are complementary.It becomes a matter of judgement and experience concerning a particular imaging application as to which methodology should be used.This must be balanced against the time necessary to prepare and process image files.In this regard ImageJ is currently more time intensive once images have been captured and processed to TIFF (or other) format.Further investigation of image pre-and postprocessing within the RTI environment may produce additional tools to aid researchers in cultural heritage, archaeology, the study of artworks and other related areas.The authors hope that this presentation will encourage others to experiment with combinations of digital imaging tools of relevance to the heritage sector.
Tablet IA5.074, Figure1, was excavated in 1953 from the site of a private house in ancient Kalhu (Nimrud, Iraq).The tablet documents a loan on security provided by a person who appeared to be a high official in the city in the time of King Ashurbanipal II (668-627 BCE).The document we image can be dated to 650 BCE.The text has been translated by D. J.Wiseman (1953) p 135, 136 &  142.

Figure 1 :
Figure 1: Small cuneiform tablet (clay) IA5.074 dim 52×35×23 mm.Of particular interest are the imprints of two seals, placed side by side on the obverse of the tablet.These are below three lines of text and a further four lines of text follow below and around the tablet.The text continues on the reverse and includes statements of witness of the loan.

Figure 3 :
Figure 3: The multi-light illumination dome for capturing RTI images.

Figure 4 :
Figure 4: Workflow for the processing of fixed camera/variable lighting angle images through ImageJ and Reflectance Transformation Imaging.

Figure 6 :
Figure 6: TIF image selection for ImageJ processing based on light directions within the RTI dome, showing NW quadrant illuminations.

Figure 7 :
Figure 7: Comparison of ImageJ processing and AR using a PTM reflectance model SW bias lighting.

Figure 9 :
Figure 9: Comparison of ImageJ processing and AR using an HSH reflectance model.NW bias lighting.

Table I
Comparing ImageJ and RTI