A Collaborative Artefact Reconstruction Environment

A novel collaborative artefact reconstruction environment design is presented that is informed by experimental task observation and participatory design. The motivation for the design was to enable collaborative human and computer effort in the reconstruction of fragmented cuneiform tablets: millennia-old clay tablets used for written communication in early human civilisation. Thousands of joining cuneiform tablet fragments are distributed within and between worldwide collections. The reconstruction of the tablets poses a complex 3D jigsaw puzzle with no physically tractable solution. In reconstruction experiments, participants collaborated synchronously and asynchronously on virtual and physical reconstruction tasks. Results are presented that demonstrate the difficulties experienced by human reconstructors in virtual tasks compared to physical tasks. Unlike computer counterparts, humans have difficulty identifying joins in virtual environments but, unlike computers, humans are averse to making incorrect joins. A successful reconstruction environment would marry the opposing strengths and weaknesses of humans and computers, and provide tools to support the communications and interactions of successful physical performance, in the virtual setting. The paper presents a taxonomy of the communications and interactions observed in successful physical and synchronous collaborative reconstruction tasks. Tools for the support of these communications and interactions were successfully incorporated in the “i3D” virtual environment design presented


INTRODUCTION
The motivation for the work presented here was the ambition to reconstruct fragmented cuneiform tablets.Digital repositories such as the Cuneiform Digital Library Initiative (CDLI) (CDLI, 2017;Lewis and Ch'ng, 2012) and the Cuneiform Digital Palaeography Project (Woolley et al., 2002(Woolley et al., , 2001;;Arvanitis et al., 2002) make examples of richly annotated photographs of cuneiform tablet fragments and cuneiform script, available to wide populations.Excavated cuneiform tablets are typically fragmented and their reconstruction poses a considerable challenge.The inadequacies of early excavations and the international dispersal of artefacts is well accounted including, for example, by Sir Wallis Budge (1925), keeper of Egyptian and Assyrian Antiquities at the British Museum from 1894 to 1924.Many thousands of cuneiform tablet fragments are now distributed within and between international collections, to which there is necessarily limited access.
Unlike jigsaw puzzles of thousands of pieces, which computers can now easily solve (Aron 2012), reconstruction of complex free-form 3D artefacts that can belong to any of many complete or incomplete "puzzles", poses a significant challenge.
A landmark paper entitled "The Virtual Archaeologist" (Papaioannou and Theoharis, 2001) A Collaborative Artefact Reconstruction Environment Woolley • Ch'ng • Hernandez-Munoz • Gehlken • Collins • Nash • Lewis • Hanes 2 proposed a system for the reconstruction of monumental fragments and described geometrical matching methods with good fragment matching performance.The computer-aided reconstruction of fragments has been an active area of research in the intervening years, though most published work has been specific to the joining of broken pottery (potsherds) (Kampel and Sablatnig, 2004).Promising results have been achieved for free-form 3D reconstruction of archaeological fragments (Huang et al., 2006) (Belenguer and Vidal., 2012), however, these approaches produce a high incidence of false-positive matches for cuneiform tablet fragments (Collins et al., 2014).Automated methods, tailored to cuneiform fragments (Ch'ng et al., 2014) have been proposed that have lower falsepositive matching (Collins et al., 2014).In common with other approaches, these methods are computationally demanding and, like all automated methods, still ultimately require human join verification.
A survey of computational reconstruction methods (Willis and Cooper, 2008), comparing and contrasting approaches from simplification of the 3D problem into a 2D problem and artefact-specific methods like pottery reconstruction methods through to methods for generic free-form 3D reconstruction, observed that while some approaches are sophisticated and computationally efficient, none claim to be ready for deployment as archaeological tools.Impediments to automated reassembly, aside from the practical difficulties associated with obtaining 3D scan sets, include the difficult search problems, the lack of surface information inclusion with object geometry and, significantly, the resolution of issues associated with large numbers of false-positive matches.A generic reassembly pipeline for virtual 3D artefacts (Papaioannou et al., 2017) has produced promising results, though still human visual inspection is incorporated in the process.Adán et al., (2012) proposed hybrid human-computer efforts to refine computer and archaeological knowledge bases with geometry, texture and feature knowledge to resource match performance.Tested on sculptural fragments this approach demonstrated good performance.The aim of the work presented here however was not to attempt to extract human knowledge into a database, but instead to engage human reconstructors in citizen science and scholarly reconstruction tasks, resourcing them with tools for making joins and enabling collaboration with support for communication and interaction.
The literature relevant to virtual 3D object manipulation and construction tasks is consistent in reporting deficiencies associated with conventional computer screen, keyboard and mouse interfaces.Recommendations have been made for a variety of interface enhancements (Heldal et al., 2005;Lewis et al., 2015), ideally, selected to suit the application (Karaseitanidis et al., 2006), and to return some functional aspects of reality to the virtual setting.Recommendations include relatively simple interaction enhancement, for example, 3DConnexion's SpaceMouse™ with push, pull, tilt and twist controls, through to the substantial technological resource provisioned by immersive CAVE (Cave Automatic Virtual Environment) environments (Heldal et al., 2006;Wolff et al., 2007) with projected visual displays and user motion tracking to enable physical gestures to be interpreted and relayed to the user's surrounding virtual world.Similarly, for improving virtual collaboration there are recommendations to enhance interfaces with enriched communication support (Beznosyk et al., 2010).
While improvements in standard computer interface devices may, in the future, support improved virtual interaction, there is currently no significant movement suggesting convergence toward a particular technology.At least not toward a technology that we might expect to be included as part of a typical Internet-accessing computer or mobile computing device.Thus, the collaborative reconstruction environment design must assume only standard keyboard, screen and mouse or touchscreen type interfaces, though support for alternative interfaces could be incorporated.Although the literature reports the loss of physical engagement and speech communication as particularly significant in collaborative outcomes, in globally distributed visions of collaboration in the workplace (Patel et al., 2012) and beyond, the barriers of language and time require an alternative to verbal communication and a means of support for asynchronous effort via standard Internet-accessing computer interfaces.This presents substantial challenges in terms of tool definition and also in intuitive interface design (Blackler et al., 2010).
With the aim of identifying tools to support virtual collaboration, experimental collaborative task observations were made and physical, virtual, synchronous and asynchronous reconstruction tasks were analysed and compared.In addition, users and stakeholders were involved in a participatory design process of tool definition (Vink et al., 2006;2008) and, importantly, in defining the look and feel of these tools.The resulting collaborative artefact reconstruction environment design provides communication and interaction support for human-human collaboration as well as tools for automated reconstruction support for human-computer collaboration.

METHODOLOGY
Preliminary pilot experiments were used to inform experimental design and determine appropriate tasks and maximum task times.Experiments were then conducted with groups of participants to observe the process of reconstruction and the collaboration involved in tasks conducted physically and virtually, and synchronously and asynchronously.Participants completed workload questionnaires for each task and each group's reconstruction performance was evaluated.The Collaborative Communications and Interactions (CCIs) were observed and categorized, and their functions were identified.A collaborative environment interface was designed, informed by both the functions from the set of CCIs and from participatory design prototyping.

PILOT EXPERIMENTS
Pilot experiments were performed to inform the experimental design: to identify suitable collaboration tasks and to provide empirical estimates of maximum time per task.As anticipated, variations were observed in participant performance, perseverance and strategy.For example, some participants made methodical attempts at matching each fragment to all others, while other participants were more selective in their attempts.However, these experiments consistently demonstrated that virtual matching tasks that would not completely frustrate or defeat participants needed to be significantly simpler than the physical matching tasks.Attempting to balance the two activities led either to physical reconstructions that were too trivial for collaboration or virtual reconstructions that were intractable.Similarly, it was observed that synchronous tasks, where participants worked together in real-time, resulted in improved performance compared to asynchronous tasks, where participants worked individually in rotation.However, despite the virtual and the asynchronous tasks being the least successful, it is virtual asynchronous collaboration that is most relevant to collaboration across the Internet.It was therefore of interest to observe the differences between asynchronous and synchronous collaboration, and between physical and virtual collaboration in order to identify the tools and functions that would best support the collaborative process and, in particular, support virtual asynchronous collaboration.
The design of the virtual synchronous task required consideration.Would the participants work together in time but at different locations, or would they work together in both time and location?Working in different locations would require supporting tools, but a primary objective of the exercise was to specify these tools.Thus, participants were asked to collaborate together on one virtual task with one shared interface, but with the aim that future testing of developed tools would include synchronous virtual collaboration of non-co-located participants.
Throughout the tasks we were interested in observing collaborative communications and interactions (CCIs).To avoid missing the nuanced communications and interactions that can be conveyed by very subtle cues, we asked participants working together on the physical synchronous task to work in silence without eye contact and, instead, to gesture or interact with their hands.In comparative testing, this restriction had no discernible impact on performance and provided an easily identifiable catalogue of CCIs that would ideally be supported in a virtual collaborative reconstruction environment.In the virtual synchronous task participants' attention was necessarily focused on the shared computer screen ahead, rather than down across the larger space of a table of shared fragments.This meant that normal speech communication and interaction was needed and for this task we observed that spoken communications tended to be very clear and explicit directions and queries.This was perhaps due to the increased difficulty of the task and the limited opportunity for eye-contact when there was a need to observe the screen.

EXPERIMENTS
Twelve groups, each with three participants, attempted two physical and two virtual reconstruction tasks.Participants were instructed that fragments would make complete or partially complete tablets and that one or more pieces would not fit at all.Tasks continued until the specified maximum task time or until all participants agreed the task was complete or that they could do no more.The physical and virtual tasks are summarized in Table I.
Figure 1 shows examples of the virtual and physical tasks.The physical reconstructions were performed on a table top with fragments of inscribed broken clay tablets that were created for the task.For the virtual reconstruction tasks, sets of fragments were scanned with a NextEngine™ HD 3D scanner.Fused polygon meshes were decimated in Blender, an open-source 3D modelling application, and the Vizard™ virtual reality toolkit was used for viewing and manipulating the fragments.A simple user interface was written in Python to provide mousecontrolled fragment movement and rotation, and camera rotation about the centre point.
Maximum times for tasks were established during pilot tests.Having been instructed that fragments would make complete or partially complete tablets and that some pieces would not fit at all, the participants were attempting a puzzle without an easily identifiable end point.For example, when attempting a jigsaw puzzle we know we are finished when all the pieces have been positioned and there are no remaining holes.However, in the experimental tasks, as with reconstruction in general, artefacts may be incomplete and fragment sets may well contain a number of non-fitting (orphan) pieces such that it becomes more difficult to know when all the pieces that could be placed have been placed.
Participants were able to stop at any point when they felt the task was complete or they could progress no further.For the virtual synchronous task, a maximum time of 15 minutes was established as an appropriately long time after which further progress was not observed.In the absence of this time limit we observed significant tenacity with this task, with a tendency for participants to continue to the point of frustration without additional progress.For the physical asynchronous task, the maximum number of individual participant attempts was four.Pilot testing showed that this was ample for the task and, while some groups did not complete the task, there was no further progress beyond the time.For the same reason, for the virtual asynchronous tasks, the maximum number of individual participant attempts was three.When attempting the virtual asynchronous task, participants were given an additional 90 seconds to allow time to navigate the virtual environment to explore the state of repositioned fragments.

Tasks
Physical Virtual

Synchronous
Participants worked together to reconstruct four tablets with four orphan fragments.
Participants worked together to reconstruct a single tablet with one orphan fragment.

Asynchronous
Participants worked one at a time to reconstruct four tablets with four orphan fragments.They work alone in rotation for 90 seconds, after which they were invited to leave a helpful post-it note annotation for the following participants.
Participants worked one at a time to reconstruct a single tablet with one orphan fragment.They worked alone for three minutes, after which they were invited to leave a helpful post-it note annotation for the following participants.On completion of the experiments, participants completed NASA Task Load Index (NASA-TLX) (NASA 2003) questionnaires to provide their assessments of perceived workload derived from weighted contributions of mental demand, physical demand, temporal demand, performance, effort and frustration.

Results for reconstruction tasks
The NASA-TLX composite and weighted workload results for each task are shown in Figure 2. Increased scores for mental demand, effort and frustration were observed for the virtual tasks compared to the physical tasks, with the overall perceived workload for the virtual tasks being 33% higher than the physical tasks.This was despite the virtual task being significantly simpler (one tablet and one orphan compared to four tablets and four orphans), but is consistent with the literature regarding standard computer interface limitations on performance and consistent also with reports from the participants themselves.For example, one participant observed that using a computer mouse for the manipulation was equivalent to doing the task with one hand tied behind one's back.
Again, in Figure 2, the workload parameters are lower for the synchronous tasks compared to the asynchronous tasks with the one exception of mental demand, where a marginal increase was observed.This was possibly associated with the additional demands of real-time interaction with other participating group members in these tasks.
Overall the workloads for the asynchronous tasks are 19% higher than the synchronous tasks.Thus, participants found the virtual asynchronous task the most demanding of the four tasks and the physical synchronous the least demanding.
The reconstructions achieved at the end of each task were assessed by a team of three researchers in terms of percentage completed correctly, completed incorrectly and undone.
Figure 3 shows the performance results for all groups and all tasks in terms of these percentages (correct, undone, incorrect).All tasks begin with 0% correct, 100% undone and 0% incorrect.Ideally, the progress of each task would follow the trajectory from this starting position to 100% correct, 0% undone and 0% incorrect.The points indicate the final progress of each task for each group and, as shown, despite the physical tasks having four times as many fragments, the virtual tasks finish closer to the starting point and the physical tasks closer to the ideal end point. 6 Figure 3. Performance results for all groups and all tasks shown in 3D space with axes correct, undone and incorrect.All pointsare on the shaded plane defined by: correct + incorrect + undone = 100 Figure 4 shows the average performance results for the different tasks.The correctness of physical synchronous reconstruction being significantly better than virtual synchronous correctness (p-value = 0.016, one-sided Wilcoxon signed rank test) and the physical asynchronous correctness being significantly better than virtual asynchronous performance (p-value = 0.002, one-sided Wilcoxon signed rank test.) The box plots in Figure 5 show the variations in correct, incorrect and undone performance across the four tasks.The percentage incorrect is low across tasks showing that the human reconstructors avoid making incorrect joins independent of the task environment: physical or virtual.This is also consistent with observations that participants, whether successful or unsuccessful at making correct joins, can recognize bad joins and will avoid making them.Decreasing correct performance across the tasks (from physical synchronous, physical asynchronous, virtual synchronous to virtual asynchronous) indicates the increasing difficulty of the tasks.Participant performance was generally closer to correct completion for the physical tasks.But the large variation and low median of the virtual asynchronous task correctness, as evident in Figure 5, shows that there is a widespread inability at this task and that while some participants can perform well, the majority cannot.It is for this majority that additional tools are required.

Results for collation of collaborative communications and interactions (CCI)
Table II shows the set of 14 collaborative communications and interactions (CCI) observed across all tasks.The set defines the fundamental communication and interaction support functions that are desirable, in virtual analogue form, in an environment that supports asynchronous collaboration.As shown, the full set of CCI's were supported in the physical and virtual synchronous tasks but much less supported in the asynchronous tasks.

Results
Participatory design exercises were performed using the PICTIVE (Plastic Interface for Collaborative Technology Initiative through Video Exploration) (Muller, 1991) paper mock-up approach.Workshop participants had expertise in cuneiform, 3D systems and computer graphics, and interests in usability and system design.A structured workshop format was used first to define user requirements and then to paper-prototype an interface and set of functions.A system interface design was created using Balsamiq™ and, at a subsequent workshop, participants further refined the look and feel of the resulting interface tools.
Figure 6 shows the final "i3D" system interface designed in Balsamiq™.The CCIs (as listed in the end column of Table II) are supported in the design with annotation support and tools for "Likes", "Messages" (for direct messaging to individuals or globally) and "Invitations" (for sharing selected fragments or workspaces with selected individuals or globally).The defined design includes visible fragment tags to indicate the existence of additional 9 fragment annotations, for example, discussion threads, directed messages and interaction information (e.g., where and how often the fragment has been joined or used).In the workspace, selected fragments can be manipulated and there are tools for annotation and automated joining, and for inspecting the fit statistics of joins.The workspace can be saved (and shared by "Invitations") and joins can be submitted for verification.
Catalogue information and search criteria are shown on the right-hand side.Fragments are selected here or imported directly from file and viewed in the central workspace.A function toolbar at the top enables users to link and unlink fragments to test possible joins which can then be locked (and unlocked) together.The design also includes a reset function to reorient fragments and a view function to toggle between obverse (front view) and reverse (back view).In the left-hand window space, as shown in Figure 6, selected fragment multi-views can be displayed.The design also includes a "Geometry" tool to apply automated matching to selected fragments, as a kind of snap-to-best-fit/s tool.The correctness or surface contribution of these joins (or manual joins) can be compared with "Join statistics" and "Surface statistics".
A desktop implementation of the system based on the proposed design has been implemented with tools for fragment selection, manipulation, locking/unlocking, and support for messaging and annotation.In testing, participants using the tools had substantially improved reconstruction performance than without the tools.The implemented system is shown in Figure 7 together with a touchscreen version.Tools for automated joining (snap-to-fit) have also been developed and tested, together with a set of fit statistics, and have been successfully used to join multiple laboratoryfabricated fragments and also scans and photogrammetric acquisitions of real cuneiform tablet fragments (Collins et al., 2014(Collins et al., , 2016;;Gehlken et al., 2017).Further work is needed to integrate these tools with the automated matching tools and to further evolve the remaining tools and the environment; for example, by supporting interface personalization (Burkolter et al., 2014).

CONCLUSIONS AND FURTHER WORK
A collaborative artefact reconstruction environment has been designed, informed by experimental observation and participatory design.The system has wide potential application for the collaborative construction and reconstruction of other artefacts.
The experiments provided an opportunity to compare tasks and quantify performance, and to establish a benchmark from which to measure the effectiveness of developed tools.Ideally, supporting tools would improve performance such that collaboration via i3D reconstruction would produce performance closer to, if not better than, the performance of an equivalent physical task.
Future environments such as those discussed here could be used to combine human and computer effort for many different object-based collaboration tasks.However, to populate environments with 3D models of real objects, there is a need to simplify and automate the acquisition process.With this achieved, a mature i3D collaborative artefact reconstruction environment could crowd-source human effort from communities of scholars, researchers and interested citizen science partners, and nature-inspired software agents could work in the background in stigmergic fashion, i.e., coordinated indirectly from environmental traces (Ch'ng et al., 2013).Thus, human and computer efforts could combine again to support reconstruction, attempting joins for fragments with matching catalogue fields and using interaction data, for example, fragment join attempts or fragment-fragment and fragment-user interactions.

Figure 1 :
Figure 1: Physical and virtual fragments.(a) Physical fragments in a synchronous reconstruction task, (b) annotation notes and reconstructed fragments after a physical asynchronous reconstruction task, (c) and (d) the virtual fragments from virtualsynchronous and asynchronous reconstruction tasks, respectively.

Figure 2 .
Figure 2. NASA TLX composite and weighted component workload scores for the four reconstruction tasks; physical synchronous, physical asynchronous, virtual synchronous and virtual asynchronous.The maximum score for each of the six components (frustration, effort, performance, temporal demand, physical demand and mental demand) is 100, making the total possible maximum workload score 600.

Figure 6 .
Figure 6.The final i3D system interface design created by participatory design and incorporating CCI functionality and tools for virtual joining.

Table II .
The set of collaborative communications and interactions(CCI)