Exeter Cathedral: A colour reconstruction for use in augmented reality devices

In this paper, we present a digital craft-based approach for 3D colour reconstruction that utilises high-quality texture maps applied to low poly triangulated meshes for augmented reality applications. We present a case study based on a scanned 3D model of the Exeter Cathedral west front which was digitally painted to recreate the colours that had existed on the real building centuries ago. These colours were informed by previous research into remnant colour samples collected from the building stonework. The manual digital painting process based on a reference study of the actual colour samples from the building stonework allowed for a colour representation that is very close to the original colours that existed on the stonework. This project was developed to be used as an augmented reality application case study for the VistaAR project. In order to ensure optimal viewing experience, highly detailed textures were applied to low poly triangulated meshes that could be zoomed into by the viewer without compromising significant details ensuring a good immersive experience for the viewer.


INTRODUCTION
In this paper, we present and explain the task of digitally painting a reconstructed model of Exeter Cathedral west front by Eddie Sinclair (1995), a specialist in the protection and investigation of historic pigmentation surfaces. The task of painting the reconstructed model was broken down into four steps; The first step consisted of cleaning up the scanned model of the Exeter Cathedral to create a good surface model. This step is most critical as it ensures there are no issues further down the production pipeline. The second step is the construction of UV tiles for the 3D model. This is the most important element for the colouring and texturing for 3D models as it allowed us to divide a large scanned UV model into six different parts of the same size. The third step consisted of creating normal maps that allow for a reduction of the triangle mesh size of the 3D model whilst keeping as many details as possible. The final step consists of analysing and painting all large and small parts of the cathedral; each individual part of which has its own colour; This painting was done manually in Substance Painter.

Digital colour reconstruction
Digital colour reconstruction (DSR) is the process of painting the surface of the 2D or 3D model to bring back the look of the original or previous realworld state that may not exist anymore (Pappas et al. 2000). DSR facilitates preservation of cultural heritages site (Stanco et al. 2011). because this process can be used to bring back the colours that existed on buildings, archaeological sites or objects in the past but have since faded away. This process can also be used to visualise what the object may look like in the future by tracking the deterioration state and making informed guesses. DSR offers many advantages: a) the data is handy to be analysed and reused by site managers and academics, b) it removes the access barrier and can be integrated with recent VR and AR technologies, c) the digital data can preserve the artefact well which will not prone to the damage of time and weathering, d) the digital replication can be duplicated and shared among different stakeholders (such as museums, universities, and the general public) to open new possible applications (Rambaran-Olm 2013). 29

AIM AND OBJECTIVES
The aim of this project was to create a realistic and engaging viewer experience for a digital walkthrough of the Exeter cathedral. The key objectives were a) to do colour reconstruction for the 3D scanned model of Exeter cathedral based on true references provided by Sinclair (Sinclair no date), and, b) ensuring that the textures were detailed enough to be engaging for the viewer whilst also being optimised for AR applications.

COLOUR SAMPLING
The colours were informed by prior colour samples collected by Sinclair (1995) from the Exeter cathedral stone work. Matching the specific colours was a difficult task because the two-dimensional reference provided to us itself added another additional level of the colour's separation from the original. Additionally, the remnant colours were already faded over time and obtaining accurate colour samples of the colours as they would have looked when painted fresh was challenging. Therefore, the colours whilst informed by Sinclair's work, still lacked a level of accuracy. We used colour the sampling tool in Substance Painter which resulted in over fifty layers being generated for each part of the model that were hand textured by us and exported as a single 8K x 8K image file texture.

PRODUCTION PROCESS
The project requirement was to create the best quality visual experience for the augmented reality application. Given that the scanned model provided was quite detailed and huge in terms of file size, we tested with a number of different texture mapping techniques and texture sizes. We found that dividing the model into three parts and using two UV maps for each part gave us the best control in terms of managing the texture painting and mapping processes. For this reason, we created six 8K x 8K texture maps that were applied to each UV space respectively. We had initially started off with two scan models of Exeter Cathedral.

3D Scanned models
The first model (see Figure 1) had 30000000 faces, was extremely detailed, high quality had a very high number of triangulated meshes and was very precise. The high poly count introduced other challenges such as long scene load-up times and frequent crashes. The second model (see Figure 2) was more optimised having lower polygon meshes compared to the first model with 1853556 faces. It was, therefore, easier to work with as the digital editing processes and operations would run more smoothly requiring less scene load and wait times and was simpler to manage in Maya (Autodesk 2019) as well as other texture painting software. The third model (see Figure 3) was created by reducing the poly count on the second model to 603903 faces in order to allow for an appropriate size and was deemed to be the best compromise to be used for AR device applications with 6 -8k x 8k texture maps.

UV tile
In the third scanned model, many sections of the model had broken vertices, non-manifold geometry, and extra polygons that had to be fixed before the digital painting could start. The scale, polygon count, and file size of the model was a growing concern. To combat this, I separated the model into three individual parts (see Figure 4) as it was easier to manage. The reasoning is that if it was one model, the UV map would be very large and time-consuming to create as well as to manage. We used six UV tiles for each of the six texture maps. We separated the Exeter Cathedral model into three sections where, each section used two UV tiles which allowed the export of two texture maps per section (see Figures 5 and 6). To make sure all the textures had the same resolution, the biggest UV shells were kept at the same size. The faces and the clothing of the statues had to be separated from the walls so that they could be more detailed.

Normal maps
To create details for the final model, we used the first scanned model to generate normal maps which enhanced the details of the 3D model without having to add polygons. For this process, we utilised Substance Painter (Adobe 2022

Digital colour paint for Exeter Cathedral west front
We used some interest references as well acquire the base stone colour, tones and details and created a stone material that acted as a base replica of the actual stone. This was used where some parts of the statue were broken. and where we were unable to colour those sections. For these broken sections, we resorted to opting for the original brown making sure to layer the colour above the base. In order to ensure better control over the texture mapping process, we split each part of the model into two different UV sections and used fill layers for each colour; having a layer for each colour makes it much easier to manage and allows for control needed to change the metalness, roughness, or heightmap. We used alpha textures to recreate patterns and used a normal brush to paint in general.

CHALLENGES
The first and biggest challenge in this project was the size and detail of the 3D scan of Exeter cathedral. In order for it to be manageable, we had to decrease the size of the mesh; the face of the statues themselves was already small to begin with, so whenever we reduced the mesh detail, we would lose key aspects of the face such as the nose, eyes, lips etc. Furthermore, these statues were made centuries ago so many of the key aspects were lost or had deteriorated over time such as limbs or other body parts breaking off. Due to the combination of these issues, reducing the mesh detail meant compromising the limited features on the statue faces and form. In order to resolve this issue, we used normal maps generated from the high-resolution mesh that we applied to the lower resolution mesh.
The second challenge was the impact of the size of the UV tiles on image resolutionlarger UV tiles provide better resolution but at the cost of memory overhead due to the high-quality texture maps being applied using image projection. Given that the model was being made for an augmented reality application, the UV tiles had to be kept small; we had to separate the whole model into four separate parts which meant creating four separate UV tiles. Given the small size of the UVs, all textures had to be hand-painted to ensure good resolution rather than resorting to 2D image projection that did not work very well for UVs of this size.

CONCLUSION
In this paper, we have presented a digital craftbased approach to use high-quality texture maps in reconstructing a scanned 3D model for augmented reality devices. Given the size, detail and high poly count of the scanned model, the digital texturing and painting process proved challenging due to which we devised a more game-based approach of using a low poly model with high-quality textures and normal maps to ensure the final images appeared detailed and engaging when viewed in the VistaAR app. The digital texturing and painting were primarily done using Maya, Substance Painter, Zbrush, Photoshop (Faulkner et al. 2019) and Unity (Unity 2022) This approach can be improved further and utilised extensively for similar augmented reality and virtual reality digital heritage reconstruction applications where the scanned model sizes are very large.

ACKNOWLEDGEMENT
The research leading to these results has been funded by European Regional Development Funds -VISTA AR project (the Interreg France (Channel) England). The authors are grateful for the contributions from VISTA AR project partners, esp. EESAB for providing the scanning models, Exeter Cathedral and University of Exeter for valuable inputs with all the background information and consultancy. The authors want to thank Eddie Sinclair for her consultancy and sharing her art drawings of 2D colouring and insight into the subject. The authors want to thank the VISTA AR team at Bournemouth University for their hard work in creating the artefacts and their enthusiasm in the VISTA AR project.