Augmented Reality Centered Rapid Prototyping

This paper discusses the potential of an updated version of IRIS, a rapid prototyping framework based on augmented reality technology. It extends a previous study conducted using a previous version of the system. Although the previous system performed as well as some other prototype methods, results gathered from the previous version led to the conclusion that the system suffered from key faults such as the insufficient resolution of the camera and the lack of connection between user and prototype device. Tests of the new version of the system showed that the increased resolution of the camera used in the new system gave a major benefit to the user interaction with overall increased performance ratings. The use of a blurry background also helped the users focus more on the prototype device and made them feel more connected during the tasks in comparison with the user experience of the previous study. The disadvantages of the new version were that users still claimed to feel distracted due to a minor lag on the video displayed on screen and the real movement of the hand. In addition, the representation of the prototype in 2D was a major factor for the users not to feel completely connected to it during testing.


INTRODUCTION
Rapid prototyping typically falls in the range of a physical prototype and usually is fairly accurate and can be implemented on a component level or at a system level.(C.K. Chua et. al 1993) Rapid prototyping solutions help product designers to quickly generate physical objects and prototypes.Prototypes created with the rapid prototyping technique allow designers to swiftly evaluate and verify their product design at an early stage and to use three-dimensional representations of the design for sales, marketing and production purposes (Aleksandar Lazinica et. al. 2006).Companies that consistently "design it right the first time" and follow a path of continuous improvement in product and process development, have a formidable edge in the crucial race to market (Wheelwright et. al. 1992).This highlights the significance of rapid prototyping in the product design cycle.
Prototyping is important as it helps verify product design at an early stage.However there are problems for computer embedded products.Gill et al. (Gill S., 2003) highlighted one of the roots of these problems by using a prototyping method, the i.e Unit: The fact that the screen is separate than the prototype.In industry often a laptop is used and the interface is tested in a software form, by using the laptop's screen.During this testing a block model, i.e. a physical model that does nothing but key input, allows the users to interact with the software interface on computer screen.Gill et. al. demonstrated that this method was introducing major delay and usability problems.The reason of using this method to perform the study is that there is no straightforward procedure for integrating the software into the hardware at an early stage of the design process.Studies by Culverhouse et. al. and Wooley [Gill, S., et. al. 2008) have demonstrated that it is crucial for the companies to be able to easily change the size of the components on a prototype (especially the screen) during the rapid prototyping procedure.
Another major detriment is that prototypes created with the rapid prototyping technique, do not meet the requirements of functional prototypes, as neither the serial material nor the serial production processes are used.(Breitinger. F. 1997) In other words, the internal modification of the prototypes in order to utilize the prototyping stage electronics such as screens and buttons requires a different internal modification of the device than the one needed in the final product.Furthermore putting real screens of varying sizes into products at prototyping stage is expensive and very time consuming.
This paper examines a rapid prototyping technique based on the use of augmented reality.It eliminates the need for internal modification of a rapid prototype model for data output purposes by providing a virtual interactive screen layered on the prototype device.It extends a previous study on the same field, addressing some limitations found concerning the usability of the system.

VIRTUAL REALITY AND PROTOTYPING
In recent years, increased computational power and technological advances in monitors and cameras, have allowed the use of Virtual Reality (VR) in the field of prototyping (Sherman W. 2003).Such technologies can provide a deeper experience during the prototyping stage as they allow the users to experience prototype interaction and achieve a more intimate connection with the prototype device, providing physicality evaluation of the device and "on the device" interface emulation (including the screen).However, virtual prototyping methods and tools suffer from various fundamental issues when they are compared to physical prototyping.For example, errors such as the delay in image processing computations and the awkwardness of the users in the virtual environment (Wang G. 2002).
Furthermore, by trying to apply VR techniques to consumer level products we encounter several restrictions, due to the current limitations in VR technology.For years now, the main restrictions that VR technology suffers are the low resolution of the screens currently used on VR glasses and the relatively small field of view (Piantanida T. 1992).Projection VR on the other hand, although it provides an increased field of view, requires more maintenance due to the significant installment requirements of the equipment needed for an adequate experience (Sherman W. 2003) The significant installment requirements also render them inflexible for single person use.In such systems, during the design time, the prototype devices need to be created by making use of soft prototyping through 3D rendering programs -a lengthy procedure and an added prerequisite skill for the designer of the prototype device.The use of such programs also weakens the link between the user and the prototype device in terms of space and shape coherency, in comparison with direct manipulation of a real artifact by the human (Gnomes A. 1999)

AUGMENTED REALITY
The use of augmented reality as a prototyping tool is gaining interest, as it provides a way of blending a prototype model in an early stage of implementation with virtual functionalities, creating an integrated prototype.
One of the most prominent augmented reality techniques, tested in the past on rapid prototyping is Spatial Augmented Reality (SAR).In SAR a virtual element is being projected on a real object from a projector.Studies on the use of SAR as a rapid prototyping technique from Itzstein SV (Itzstein et.al.2011)andVerlinden JC (Verlinden JC, 2003) have proven that prototyping is possible with SAR because this technique is solving various problems related to visual quality (e.g., resolution, field-of-view, focus, etc.).However, various issues render the whole technique as suboptimal for general rapid prototyping.Some of these issues include technical problems (e.g., tracking, lighting, etc.), and human factors (e.g., cumbersomeness, etc.) (Šťastný J et. al. 2011), issues like limitation to non-mobile applications and occlusion or shadows cast on the surface by the user or the other parts of the system.

THE IRIS SYSTEM
Our approach makes use of a screen-based video see-through display.This approach provides a window to the world solution (Feiner S.1993).The screen is placed in a fixed position and angle eliminating the need for head tracking.Technological problems that these devices were suffered for years, like lack of adequate resolution from the camera and from the monitor, are being gradually resolved.Such advances mean that screen-based, transparent AR use for rapid prototyping is becoming more and more viable.
The system in its current form is based on a modified version of ARToolKit, an open source augmented reality framework written in C++ that makes use of visual markers with the use of USB web cameras in order to overlay virtual elements on real devices.The system was combined with FantastiqUI framework, a C++ implementation for low level access to Flash files in OpenGL environment.Integrating the system with a Flash interface allows designers to easily implement their software interface design (as Flash is a commonly used Graphical User Interface (GUI) design tool in industry).
The version of the IRIS system discussed in this paper is an improved approach on the version of the previous study.Based on feedback gathered from the previous study, we tried to identify usability problems connected to technological factors and we tried to address them in the hope of achieving an improved overall system performance.The main challenges addressed in the current version of the IRIS system are the low resolution of the image displayed on screen and the perceived lack of connection of the users to the represented on-screen prototype due to depth perceptional problems introduced by the monitor on which the prototype was presented.

Improvements on current IRIS system
The previous study highlighted various aspects that would benefit from further improvement.The previous implementation of the system highlighted two very significant technological problems: The lack of adequate resolution on the screen.The prototype displayed on the screen was blurry and details such as button labels and shape was very difficult to identify.
Lack of connection with the device.The users were looking at the prototype through a monitor.This 2D representation of the prototype was blended with the background and users reported feeling that their hand holding the prototype displayed on the monitor did not belong to them.
In this paper we tried to solve the above barriers to the better use of AR for rapid prototyping by introducing the following: 1) To improve image resolution, a high definition (HD) web camera was used instead of a common low resolution web camera.The resolution used for this study was 800*600 in comparison with 640*480 in the previous one.We found that due to the better quality of the camera sensor, even at 640*480 there was significant difference in contrast and clarity between the two cameras, with the HD one having crisper contrast and smoother movement.
2) To tackle perception problems, the solutions were twofold: By putting a blurry background behind the participants' hand and by asking the participants at the beginning of each study to keep their hand in an optimal place, where the size of the hand on screen was the same as the size they would have perceived of their hand if there was no screen between it and their eyes.

EMPIRICAL TESTING
In a previous study, Gill et.al. conducted a series of tests comparing the performance of a real BT Equinox phone, an Equinox / IE Unit prototype and a screen based prototype using a methodology developed by Molich and Dumas (Molich, R., et. al 2006).
16 members of administrative staff from the Cardiff Metropolitan University took part in the study.They ranged in age from 22 to 55 years.Experience of mobile phone interfaces was broadly similar to that described by Gill et al. (Gill S. 2003) in their experiments and none of the participants who took part in these trials had participated in the earlier study.In this study, we asked the participants to accomplish the same tasks as in our previous study.
These tasks are linked with common functions (turning the phone on and off) and more complicated ones (dialing a number and changing the background wallpaper).The sequence in which the users were asked to accomplish the aforementioned tasks was based on the task difficulty, in order the users to feel gradually comfortable with the functions of the system.For this experiment we asked the participants to perform the same tasks as the previous study.With the specific tasks we tried to identify the effectiveness of IRIS2 implementation on use on rapid prototyping.We also compared results with the previous implementation of the system, spotting usability improvements.Furthermore the same tasks were used in previous studies from Gill et. al. to evaluate the effectiveness of a prototyping tool called ie Unit [5].Thus by using the same tasks was easier for us to evaluate and compare the usability effectiveness of the ie implementation and the IRIS2

Procedure description
Each participant was given a questionnaire and instruction sheet.The info gathered from the questionnaire form, such as the experience of the user with the use of mobile phones, the age, etc. were analyzed for the qualitative analysis of the experiment.The users were provided with a basic description of the interface used for the study and they were allowed to do any questions they needed in order to feel more familiarized with it.
According to the feedback gathered from the previous study, there were two major changes in the new implementation of the device: After experimentation an optimal distance was found between the system camera and the hand, so that the hand appeared the same size onscreen, as perceived by the user in real life.
The performance of participants was converted to four different interval data per task.These intervals included 0= success, 1= minor, 2 = serious, 3 = catastrophe.Analysis of performance outcome and performance time used a 5 (device type) x 4 (phone task) mixed analysis of variance (ANOVA).• Turn the phone OFF Figure 1 illustrates the mean time taken to complete each of the four phone tasks as a function of device type.There was a significant main effect of device, F (4, 106) = 23.6,p < .001, a significant main effect of task, F (3, 318) = 159.75 and an interaction between device and task, F (12, 318) = 7.31, p < .001.To explore the main effect of device, a series of pairwise post hoc tests (REGWQ) were performed.These showed that there were reliable (p <.05) differences between software/IRIS and IE unit/Equinox/IRIS2 and also a reliable (p <.05) difference between IRIS2 and all the other devices.None of the other pairwise comparisons were significant (p > .05).
Figure 2 shows the success outcome (rating) in completing each of the four phone tasks as a function of device type.There was a significant main effect of device, F (4, 106) = 10.24,p < .001, a significant main effect of task, F (3, 318) = 32.48 and an interaction between device and task, F (12, 318) = 5.81, p < .001.To explore the main effect of device, a series of pairwise post hoc tests (REGWQ) were performed.These showed that there were reliable (p <.05) differences between software and IE unit/Equinox/IRIS 2.0 and also between IRIS 1.0 and IE unit/Equinox/IRIS 20.None of the other pairwise comparisons were significant (p > .05).
From the results it is evident that IRIS 2.0 outperformed IRIS 1.0 on the first, second and fourth task.The reason that it did not outperform IRIS 1.0 on the third task is linked to changes in user interface trends and is discussed hereafter.It should be noted that even though on the first study the performance of the system was lower than IE Unit and Equinox, the improved version of the second study placed them on similar usability levels.Nevertheless, it is clearly demonstrated that IRIS 2.0 performed remarkably well on the first task as the performance time/rating where in both cases high, outperformed only by the real Equinox device concerning the mean rating.

Task 1:
Users were asked to turn the phone on.There was no guidance on where the power button was located and thus from observing the time needed for the users to detect the power button, we were expecting to evaluate the effectiveness of the new high resolution camera used for the study.In the previous study the limited resolution prevented the users from identifying the button, spending a fair amount of time trying to switch on the device by pressing buttons on the keypad.With the use of a high definition camera, during the current study, we managed to improve to a greater extent the clarity of the image displayed on the screen.Users were able to spot the On/Off button almost instantly as the improved contrast and resolution made it almost look three dimensional.Some of the users spent a bit more time trying to turn on the phone by continuously pressing the "end call" red button on the phone's keypad.Comments like "I used to turn my phone on by pressing the power button" indicate that users were accustomed to turning the phone on and off this way from their personal mobile phones.

Task 2:
During the second task, users were asked to dial a specific telephone number provided to them.When the users started the task, the phone was displaying the main UI screen.The users would need at that point to start typing the number and when they finished they were asked to press the green button to perform the call.
In our first experiment, the represented numbers of the keyboard on the screen, lacked the high resolution the camera we used on our current study provided (Figure 4).Participants were able to distinguish easier the numbers on the keyboard.This was reflected on the better timings we recorded during this task.Also the number of mistakes the users did during the typing of the numbers was minimal.Between them the 16 participants made four mistakes entering the numbers, in comparison with nine mistakes in the previous study.

Task 3:
During the third task, users were asked to change the background photo of the phone.The results in this task are quite interesting and highlight UI usability problems beyond the scope of our study, which is mainly concerning IRIS System.
The users were asked to navigate through the phone's menu until they find the option corresponding to "background customization" and then change the photo.Even though the users were able to quickly navigate through the menus of the phone, the way the menus were represented on screen confused most of the users, forcing a considerable number of the participants to abandon this task before completion.The main problem was that the customization section of the phone was represented with a music note icon (Figure 3).As the device that the prototype was representing was quite old, mobile phones did not used to be used as music players.With the intuitive new generation of smart phones, like Apple's iPhone, Research in Motion's Blackberry, the Sony Ericsson Xperia, Nokia Lumia and Samsung Galaxy, people are becoming more familiar with multimedia devices and the idea that a music icon represents a music player rather a customization section for wallpapers and ringtones.
It is interesting to check the results from the previous study.We will find out that during the same task in the previous study, users did better on finding the background customization section.In this study almost half of the users completely failed to find it.This fact reimburses our opinion of people getting used with different representation of the same functions as they are using devices of different technology.

Task 4:
During the fourth task the users were asked to turn the phone off by pressing the same button they pressed to turn the phone on the first task.We were expecting the timings to be better than the ones of the previous study because of the introduction of the high resolution camera.Our expectations were right as the timings indeed were better than the first study.The users were able to almost instantly turn the phone off.

DISCUSSION
From the results we can clearly see a vast improvement in the timings of turning the phone on and off and also on the calling a number task.It is evident the introduction of a higher resolution camera helped the users quickly identify interactive elements on the surface of the prototype.
Although the system outperformed the previous version, users still felt that using it was not the best experience for them.The system seems that has a learning curve, as judging by the comments of the users, they initially tended to feel uncomfortable and nervous when they started the study before the first task but their comments after the fourth task showed enthusiasm and satisfaction from the overall experience.
Users felt much more connected with the representation of their hands on the screen while using the IRIS 2 in comparison to the experience they had with IRIS 1.The minor, milliseconds lag between the movement of their hand and the represented one on the screen in combination with the lag of depth perception due to the 2D monitor representation of their hands on IRIS 1 made the users feel quite disconnected concerning the representation of their hands on screen.This is highlighted by comments like: "There seems to be a lag between my real fingers and the ones on the screen", "I am not completely sure for the distance between my finger and the phone keys".In IRIS 2 users were asked to use the devices in a fixed position behind the screen where the representation of their hand was approximately the same size of their hand.We also placed a blurry background behind their hand of random colors which improved even more their experience.In sum, according to their comments and reactions the connection they felt with the system was much better than the experience of the users to the previous implementation of the system.Some users were even excited as they found the experience unique and appealing.
Another problem with the IRIS system was the covering of the augmented reality sticker (bar code) with the user's hand (when switching the device on and off), which resulted in the GUI not being transposed onto the physical model.An alternative solution to putting the interface over the sticker, could be to place more than one sticker in random places on the surface of the prototype, and thus decreasing the chance all the stickers to be overlapped by the fingers at the same time.That could allow a better visual tracking of the markers and a more effective translation of the position of the prototype device on space.Furthermore.use of chroma keying techniques could solve the problem of displaying the fingers in front of the screen.Something that could potentially lead to extend the prototyping capabilities of the system for testing touch screen devices.
The real BT Equinox phone • IE Unit: The prototype phone using the IE Unit and the GUI displayed on a separate PC monitor • Software: The screen based prototype • IRIS: The physical model using the augmented reality technology of the IRIS system • IRIS2: The upgraded IRIS system.Tasks: • Turn the phone ON • Dial a given number • Change the background photo