Reflectance Transformation Imaging Systems for Ancient Documentary Artefacts

This paper discusses the interim results of the AHRC RTISAD project. The project has developed and tested a range of techniques for gathering and processing reflectance transformation imaging (RTI) data. It has also assembled a detailed understanding of the breadth of RTI practice. Over the past decade the range of applications and algorithms in the broad domain of RTI has increased markedly, with current working addressing issues such as large resolution capture, 3D RTI, annotation, enhancement amongst others. Capture of RTI datasets has begun to occur in all aspects of cultural heritage and elsewhere. This has in turn prompted the development of policies and methods for managing and integrating the large quantities of data produced. The paper describes these techniques and issues in the context of a range of artefacts, including painted Roman and Neolithic surfaces, examples of ancient documents in a variety of forms


INTRODUCTION
The Reflectance Transformation Imaging Systems for Ancient Documentary Artefacts project was funded by the UK Arts and Humanities Research Council in 2010 via the Digital Equipment and Database Enhancement for Impact (DEDEFI) scheme.Reflectance Transformation Imaging (RTI) describes a suite of technologies and methods for generating surface reflectance information using photometric stereo i.e. by comparison between images with fixed camera and object locations but varying lighting (Woodham 1980).RTI now describes a file format (Mudge et al. 2006) in addition to a set of methods.The most common implementation of RTI is via Polynomial Texture Mapping (PTM) invented by Tom Malzbender of HP Labs in 2000 (Malzbender et al. 2000(Malzbender et al. , 2001)).Subsequent work has identified alternative approaches to compress defined surface properties including spherical and hemispherical harmonics (Mudge et al. 2008).Applications of PTM and other approaches to archaeological data have been trialled in a wide range of contexts (Earl et al. 2010a) and this project aimed to produce a broad understanding both of the state-of-the-art in RTI and its future potential, and the implications for its applications to ancient document and other archaeological data.
The aims of the Reflectance Transformation Imaging Systems for Ancient Documentary Artefacts project were as follows: (i) Produce two RTI capture systems (ii) Create UK RTI capture and research hubs at Southampton and Oxford (iii) Pilot capture of a range of ancient document artefacts (iv) Pilot capture of a range of archaeological artefacts (v) Integrate RTI into existing practices (vi) Prototype an archaeology RTI repository (vii) Disseminate widely for academic and public impact Over the last year of research we have met all of these aims.Specifically, we have produced a number of dome-based capture systems and trialled novel technology provided by our partners at HP Labs.We have also established Southampton and Oxford as hubs for RTI research and implementation, including embedding RTI in the teaching and research activities in the host institutions and developing sustainable plans for long-term data capture and development.We have captured a broad representative sample of the ancient document and archaeological artefacts identified to be appropriate for RTI study.In addition we have developed and applied tools to embed RTI capture and processing in existing approaches, most notably in the context of annotation.We have developed a prototype repository for RTI data in collaboration with the Archaeology Data Service and Cultural Heritage Imaging (CHI) which is leading to further developments in the documentation and management of RTI datasets.Finally, we have undertaken widespread dissemination reaching academic, industrial, local and national government, the general public and other stakeholders all over the world.

RTI CAPTURE TECHNOLOGIES
The RTI literature includes many alternative capture approaches.These can loosely be divided into highlight and lighting rig systems.This project made extensive use of both types, and developed examples of each.

Lighting rigs
Fixed lighting rigs enable consistent capture of surface imaging data, with a fixed position and direction of light sources and means to calibrate colour variation between them.Lighting rigs have been developed in the form of arcs and arms, geodesic and hemispherical domes, and enclosing spheres.We developed a range of dome-based systems for this project, at varying radii.The radius of the dome defines the size of the object that can be captured to sufficient quality and also the types of lens needed and hence any distortion present.The large format domes produced by the project are 1m diameter and are made from plastic, divided into four quadrants for transportation.The size enabled high quality capture of the main size of target objects.Anything 17cm across or smaller can be efficiently imaged, including small inscriptions, stylus and lead tablets, large cuneiform tablets and many archaeological objects such as stamped brick and amphorae.Each dome has 76 fixed LED light sources.The plastic dome is painted black, with a matte interior to minimise spurious illumination.The aperture at the top of the dome allows movement of the camera in order to generate pairs of data for photogrammetric processing.The project uses Nikon D3X DSLR cameras.These provide 24.5M pixels with a 14 bit 35.9 by 24.0mm FX CMOS sensor.The resulting images are 6048 x 4032 pixels.The range of lenses employed includes 35mm, and 50mm and 105mm macros.We have seen no need to remove lens distortion as a standard processing step.However, we have identified suitable batch processing tools to employ where necessary.The dome lighting system has been trialled with several alternatives including Cree and Osram LEDs.The CREE XREWHT-L1-0000-008E7 produces 74 lumens while the Bridgelux BXRA-N0402 produces 400 lumens which helps reduce exposure times.These produce a useful white light, have very fast on/off time, high consistency and negligible IR/UV emissions.A custom USBcontrolled switching unit and constant current source is driven using text commands from the capture software.Typical exposure was 0.25s at F8 so objects are only illuminated for around 20s.
There can be colour variations between some LEDs and their colour temperature is fairly low so we have developed a batch colour calibration workflow which produces sRGB.This uses the VIPS and NIP (Martinez andCupitt 2005, Cupitt andMartinez 1996) open source image processing environments which produces easily repeatable processing workflows.In addition they provide an audit log of operations which could easily be fed into the RTI process metadata.We place a Macbeth colour calibration chart on the capture stage in order to generate a batch conversion colour calibration matrix for each light.Figure 3 shows the type of colour cast that is removed.One aim of the project was to produce a system that could be used continuously for large collections.As a consequence the dome system incorporates a scissor lift on a sliding platform which delivers the sample to the required location and height beneath the dome.In the current configuration the project captures a full set of images in c. 2 minutes, with an anticipated speedup as we introduce software optimisation in the final part of the project.Processing can then be accomplished alongside the next capture cycle, in advance of it, or in a batch process overnight.It is also possible to resample the captured images in order to produce a quick indication of the data quality.

Figure 4: Example of delivery mechanism
A secondary aim was to produce a set of tools that were portable, and in particular that could be employed without prior planning for RTI capture.To this end we have experimented and further developed applications of the highlight RTI method (see below).In addition, we have developed a portable miniature RTI dome 25cm in diameter that makes use of a smaller camera and requires only a netbook or similar device to power and manage its operation.

Highlight systems
The highlight RTI method enables rapid capture of data without a fixed lighting rig (Mudge et al. 2006).This project has used highlight in a range of contexts, including field capture of rock art and graffiti (Mudge et al. 2010), museum capture of small finds, and microscopic capture of conservation features.
The application of the technique to microscopic features has proved particularly exciting.The highlight RTI technique works in an identical fashion when using a microscope and small highlight target (such as a ball bearing).In one recent case study we imaged a ceramic roundel/ button from Tomb A at Derveni, Greece, located in the Archaeological Museum of Thessaloniki.It has a diameter 2.3cm with remains of a red colour.The PTM enabled more detailed condition assessment, and in particular examination of craquelure, flaking and loss of red colour layer, salt efflorescence and depositions on the surface, as well as the rough ceramic texture.

RTI processing
The PTM approach was designed to provide a quick, light-weight compression of the reflectance and hence shape of imaged surfaces.The intention is not to produce accurate surface normals but rather (in terms of graphics) to produce enhanced surface simulation and (in terms of surface analysis) and intuitive means for exploring surface shape.There are a wide range of alternative methods for reconstructing more robust surface normal from similar input datasets (Drew et al. 2009).There have also been a number of metric analyses of PTM and comparisons to other forms of data acquisition (Dellepiane et al. 2006, Macdonald andRobson 2010).
However, in our experiments we have seen that the quality of data produced allows for useful comparison of surface normal (for example before and after conservation processing) and definition of significant surface deviations.We found that surface normal manipulation tools used in standard computer graphics pipelines work as efficient means to register and compare outputs.The potential for RTI datasets as the input to nonphotorealistic rendering (NPR) was identified by Bartesaghi (et al. 2005).More recently Toler-Franklin (et al. 2007) has demonstrated the broader potentials of NPR using red, green, blue, normal (RGBN) datasets.In our own work we have used surface normal contours to enhance painted surfaces and applied standard image processing tools such as sobel filters to enhance RTI data.It is clear from our applications that it would be very simple for conservation and other specialists to incorporate RTI image processing into their existing workflows, to considerable affect.

RTI management
In addition to the interpretation of RTI data the project has explored means to link interpretation to the data, to share these interpretations, and to manage the production of long-term repositories of RTI information.
Our extension to the newly updated RTI Viewer (RTIViewer) allows annotations to be created.These annotation boxes link to a textual description (and hence to external datasets) and also describe the viewer settings at the time the annotation was created.These bookmarks then enable the viewer to cycle through the annotations with the viewer updating to indicate the conditions under which the annotation was made, including the lighting direction.In this way it is possible to provide a nuanced representation of particular interpretations.As the annotations are stored as an external XML file they can be shared with others, combined, and re-imported to identify alternative readings.

Figure 10: RTI viewer showing an annotation
The project has also considered the long-term sustainability of RTI datasets building on strategies identified by CHI (Mudge et al. 2008).A repository has been established hosted by the Archaeology Data Service which enables access to raw RTI input data, processed results, and metadata associated with their production.However, delivery of RTI data via the web remains problematic.The large file size has usually required down-sampling of RTI datasets for public dissemination.For the RTI repository we are taking advantage of two innovations.First, the RTI Viewer enables streaming of pyramidal datasets over the web.In addition the SpiderGL implementation of PTM (Di Benedetto et al. 2010) employs a different quadtree structure to enable rapid interaction with very large RTI datasets.

RTI CAPTURES
The project has attempted to trial RTI and to document its value on as wide a range of materials as possible.We will publish a separate evaluation of each of these applications in the future, including expert input from domain specialists.For this paper we provide a brief summary of the perceived value in two domains: cuneiform and Egyptian graphical culture.

Cuneiform captures
Cuneiform documents present a wide array of problems for traditional image capture.The inscribed objects are three dimensional; the inscribed surfaces are rarely flat, but most often convex; writing and non-writing marks may be found on any surface, often in multiple layers and directions; writing can be deeply or shallowly impressed; and the seal impressions often leave a very shallow relief of both text and image across text or below it.Furthermore, cuneiform texts are inscribed on both stone surfaces and objects made of clay or gypsum: the clay may be fired (mostly after discovery), and can be either matte or shiny.Earlier studies have demonstrated the utility of RTI on cuneiform and our intention has been to update these studies based on current capture methods and to undertake a detailed analysis of the utility of the techniques, and potential enhancements.
From the viewpoint of the cuneiform specialist, the images produced in this study provide a considerable step forward in digital representation of cuneiform texts.First, the software conforms to standard practice and enables easy creation of an idealised snapshot of each tablet, using one or more lights perfectly to illustrate a given point.With online viewing software still in its infancy (see above) the Cuneiform Digital Library Initiative (CDLI) has opted for the export of such stills for online display of the images.However, our project's addition of annotation tools which can re-set any desired parameters further increases the potential of distributing the RTI data alongside such snapshots.
Cuneiform scholars are faced by very large collections of often decaying documents of which only roughly 20% of the total global holdings of probably in excess of a half million clay tablets and fragments have been published during the last 150 years of study.As a consequence our project took the pragmatic approach of employing a dual track capture process, using a flatbed scanner for regular surfaces and the RTI dome for unusual and difficult surfaces.Timings for this process were c. 5 minutes capture time using the flatbed scanner and c. 25 minutes using the RTI dome, capturing in both instances six surfaces of each object.This combined method appears to best suit the needs of cuneiform scholars.We anticipate that the method outlined by Brown et al. (2008) for obtaining RGBN data from a calibrated flatbed will very soon provide us with an efficient middle ground.

Egyptology captures
From an Egyptological perspective, in addition to improved readability for inscribed/ decorated objects, the value of RTI also lies in the detailed surface information provided through different RTI enhancement options.Close study of the RTI image of a perforated ivory with the aid of the specular enhancement and diffuse gain settings, has uncovered palimpsest and clarified marks relating to surface preparation and tool slips (Piquette forthcoming).The enhanced visibility RTI provides for such surface transformations, including information about tool use and technique, is of paramount importance for exploring material processes of script and image production and the reconstruction of past individual and collective practice.

GRAPHICS APPLICATIONS
Elsewhere (Earl 2010b(Earl , 2009) ) we have demonstrated the possibilities of using RTI data in the construction of computer graphic simulations, in part through the creation of virtual PTMs.The potential of PTM for rendering was a prime motivation behind its development and enhancement (Hel-Or et al. 2003).During the current project we have concentrated on developing the use of diffuse and normal values from the captured RTI datasets to enhance simulations of the captured surfaces in novel environments.This has included the simulation of painted wall plaster at the site of Çatalhöyük in Turkey.In addition to the creation of digital representations the RTI data themselves have proven of value in communicating information about surface morphology to a wide audience.For example, at the Fitzwilliam Museum (Bridgman and Earl in press) we have been able to produce RTI captures that demonstrate the complex surface behaviour of lusterware ceramics, and in the future we will enhance this with HSH processing.Other dissemination has been via a range of conferences, workshops and lectures.These have included the Portable Antiquities Scheme, the IFA Finds Liaison Group, museum specialists, Archaeology and Classics students, English Heritage, the National Trust and the British Library (including demonstrations of the technology as part of the Growing Knowledge exhibition).Public dissemination has included articles in the Economist and British Archaeology, and a feature on a BBC history documentary.

CONCLUSIONS AND FURTHER WORK
The project has demonstrated the considerable potential of RTI in a wide range of subject domains pertaining to cultural heritage, and indeed more broadly.Our review of the state-of-the-art suggests a thriving research community characterised by a refreshing degree of collaboration and openness.It is our hope that future work further to enhance and integrate the technologies, and to undertake exhaustive and extensive captures will both lead to even greater awareness and democratisation of this suite of technologies.

Figure 1 :
Figure 1: Robotic RTI arc in use at Herculaneum

Figure 11 :
Figure 11: RTI detail of ivory funerary label showing erasure marks underlying incised signs in the lower left.Diffuse gain (top), specular enhancement (bottom).From Abydos, Egypt, c. 2700BC (EA 32668 © Trustees of the British Museum)

Table 1 :
Summary of RTI captures to date