Challenges of using stereoscopic displays in a touch interaction context

This work examines how common use scenarios for touch interactive stereoscopic displays might exacerbate visual fatigue. We identify technological constraints of current stereoscopic displays and image separation techniques as the potential underlying cause and generate a set of hypotheses concerning the implications for end users. Furthermore we outline a proposed study to examine these hypotheses. stereoscopic display, touch interaction, physiological effects, study design


INTRODUCTION
In recent years advances in capture, editing, broadcast and display (Holliman et al. 2011;Meesters et al. 2004) have led to a widespread uptake of stereoscopic displays within households.More recently, with the broad adoption of multi-touch display interfaces (tablets, smart-phones and multitouch tables), a number of researchers have begun to explore the potential of utilising stereoscopic displays in conjunction with co-located direct touch interaction in novel ways (Valkov et al. 2011).
In this paper we discuss the design of a study which aims to examine whether these usage scenarios result in negative physiological effects and what factors, relevant to interaction design in these usage scenarios, are of most concern.There is a wealth of research exploring issues of human factors in the use of stereoscopic displays.In particular one issue that has received significant focus is the potential for negative physiological effects induced by viewing stereoscopic images such as visual fatigue.

STEREOSCOPIC MULTIPLEXING DISPLAYS
Stereoscopic displays work by presenting separate images to each eye of the same scene but from a  slightly different perspective.The resulting images take advantage of our natural ability to perceive depth through binocular vision.The most common commercial form of stereoscopic display, and the form we focus on in this paper, relies on multiplexing to display independent images to each eye using a single display (figure 1).
There are various approaches to multiplexing.Fundamentally each approach attempts to display separate images to each eye through a single display utilising passive or active filtering techniques (figure 2).For an overview of stereoscopic multiplexing see Holliman et al. (2011); Meesters et al. (2004).
In order to provide context for the following discussions we briefly outline the various multiplexing approaches used: Colour multiplexing uses a different colour lens over each eye, typically at extremes of the visual spectrum (red and blue) to act as a filter to distinguish between two overlaid images that are separately coloured.This technique is limited in effectiveness and results in significant cross-talk between the images received by both eyes.However, it is still commonly used as it is cheap and works with a wide variety of existing displays.
Polarization multiplexing uses polarising filters in both the display and in front of the viewers eyes to perform image separation.This approach is superior to colour multiplexing as it is less likely to lead to crosstalk between images and has less negative impact on colour perception.However, this technology requires a purpose built stereoscopic display with suitable polarising filters and results in reduced image contrast.
Active shutter lenses utilise time-sequential multiplexing.The display quickly alternates between images intended for the right and left eye.The viewer is required to wear shuttering lenses that are synchronized with the display to block the images to alternating eyes.This approach overcomes many of the issues associated with passive multiplexing techniques but requires a display with a high refresh rate to reduce perceived flickering (> 120Hz).The lenses are more expensive and heavier, so less comfortable to wear, than passive lenses.
Auto-stereoscopic displays do not require the user to wear any form of lens to perceive a stereoscopic effect.They utilise a barrier or lens over the display to restrict the incident angle at which an image is perceived.Therefore the left eye perceives, given a slightly different angle of incidence, a different image to the right eye.This approach requires two images to be broken into small chunks and displayed in separate viewing zones which significantly reduces the overall resolution of the display.
The common feature of these approaches is that they all: • rely on a single display and some form of image multiplexing.
• degrade in perceived performance as the user moves from the optimal viewing position.

STEREOSCOPIC IMPAIRMENTS
Stereoscopic impairments are a result of inconsistencies in the images being received by the eyes based upon natural binocular vision.Some of these impairments are due to the configuration of the image capture apparatus (such as camera angles and relative positions).Others are due to technological limitations on how the visual medium is created and broadcast (video file compression algorithms, planes of depth (Yamanoue et al. 2000)).See Meesters et al. (2004) for more detailed overview of stereoscopic impairments.In this paper we concentrate on those stereoscopic impairments introduced as a result of the nature of the multiplexing display technologies.
Stereoscopic impairments often result in negative physiological effects such as visual fatigue, eye strain, headaches and dizziness.Although these are unpleasant for the user they may not be immediately obvious to the user during interaction.
A commonly used objective measure of the effects of stereoscopic impairments, as a result of visual fatigue, is an observed decrease in performance of the vision system and is often associated with visual discomfort (Lambooij et al. 2009).This can be caused by a number of factors.

Viewing angle
Stereoscopic display technologies are typically designed to have a single viewing position or range where the stereoscopic effect is optimal (this is usually orthogonal to the horizontal and vertical planes of the display).However, when in use the angle which displays are viewed can vary markedly depending upon context of use.
In the common household set-up viewers are likely to vary their viewing angles horizontally more so than vertically.This is reflected in the design of 3D displays such that the stereoscopic effect is fairly robust to variance in horizontal viewing angles but moving away from the optimal viewing angle in the vertical direction causes a significant degradation in the perceived stereoscopic effect.This degradation most commonly occurs due to increased crosstalk.
Crosstalk occurs when an image intended for one eye is unintentionally displayed, in full or in part, to the other eye (figure 3).All technologies that utilise multiplexing to display stereoscopic images have the potential to introduce crosstalk.The extent and impact of crosstalk is dependent upon the image separation technology utilised (Woods 2010).
Crosstalk is associated with negative physiological effects (Pala et al. 2007) even at very low levels (Pastoor 1995).A second effect induced by change of viewing angle is sheer distortion which is observed as an unnatural movement of a visual object.In the natural world, as we move around a stationary object the object appears to remain stationary.However, in stereoscopic displays sheer distortions make objects appear to move either with, or in opposing directions, to the user as the user moves relative to the screen.Any viewer moving outside of the optimal viewing point are prone to experiencing distortions in perspective which makes the perceived 3D image seem unnatural (Woods et al. 1993).

Viewing distance
Viewing distance is also an issue in the use of 3D displays, because of the twin actions of accommodation and vergence in the vision system (Purves and Lotto 2003).Accommodation is the process of focusing the eye on a target in the visual field.Vergence is the process of moving the eyes in opposing directions to ensure some object is exposed to the centre of the retina where vision is most acute.These two processes are intrinsically linked and form a natural reflex in our vision system.
In normal vision these are key factors used in depth perception.However stereoscopic displays create an artificial visual scene which breaks this link.Since our eyes focus on the display, which is at a fixed distance, then accommodation is constant.However the vergence varies depending on the image that is being displayed.
This behaviour is unnatural and is exacerbated when the display is closer resulting in increased binocular parallax (Emoto et al. 2005).There is some debate in the literate as to what role this plays in the onset of visual fatigue whilst viewing 3D displays but it is clear that vergence-accommodation is negatively affected by stereoscopic displays (Yano et al. 2004).Lambooij et al. (2009) point out that there is a depth of focus (accommodation range) within which "comfortable vision" can be achieved and that this range reduces with proximity to the screen.

IMPLICATIONS FOR INTERACTION
It is clear that if stereoscopic displays are viewed outside of their optimal viewing conditions that there is potential for negative physiological effects upon the viewer.
TV manufacturers develop stereoscopic displays to perform optimally in their intended context of use in order to alleviate these issues.This is typically to account for variance that would occur in a normal household viewing context (wall mounted at eye level for a seated viewer facing an opposing wall) with viewers looking directly at the screen.Modern displays are optimised for minimum viewing distances that correspond to a maximum visual field of view of 40 • .For a 60 screen this equates to approximately a 6 minimum viewing distance.
Yet these design assumptions may be stretched or broken by their use in novel stereoscopic touch interactive displays.For our study we will consider the scenario of a stereoscopic display mounted horizontally and combined with touch interaction in the form of a stereoscopic multi-touch table.Based on the extant literature we make the following hypotheses.
H1: If the user is within arms reach of the display then visual fatigue will increase due to enhanced disparity of vergence & accommodation.
In the touch table scenario a user will be required to be within arms reach of the display.This places implicit constraints on viewing distance and thus raises potential issues with vergence & accommodation.
H2: If the user is viewing a display placed horizontally in a table configuration then visual fatigue will increase as a result of more oblique viewing angles increasing the possibility of crosstalk in the perceived image.
Viewing angles will inherently be increased due to the position of the user relative to the screen as depicted in figure 4. In particular users can experience increased viewing angles in both the vertical and horizontal directions (relative to the screen).This has the potential to significantly increase crosstalk.

Participants
Participants will be recruited by advertisement in the local community.A gift voucher will be offered to each participant as an honorarium for participation.Participants will be required to have fully corrected vision and to have no known stereoscopic deficiencies.No other constraints will be placed upon selection.

Independent Variables
The study will consist of two independent variables: • Orientation of the display: Horizontal and Vertical.
• Distance of viewer from display: Within touch range and Within optimal design parameters.
The experiment will be run as a reduced factorial between-subjects design.The three conditions will consist of: • Participant is within touching distance of a horizontal display.
• Participant is within touching distance of a vertical display.
• Participant is an optimal distance (6') away from a vertical display.

Dependent Variables
Visual fatigue can be measured objectively using a number of observable changes in the performance of the vision system directly.However these direct observations are difficult to undertake requiring significant expertise and costly equipment (Lambooij et al. 2009).Reading performance has been shown to be a good objective indicator of visual fatigue since reading performance (in terms of efficiency and effectiveness) tends to decrease with increased fatigue (Lambooij et al. 2012).
For this study participant reading performance will be measured using the the Wilkins Rate of Reading Test (WRRT) (Wilkins et al. 1996).Subjective fatigue will also be measured using a survey developed by J. E. et al. (2003) to distinguish between levels of visual discomfort.

Procedure
The experiment will be conducted in a room with controlled lighting using an active shutter stereoscopic display mounted to a stand that can be transformed between horizontal and vertical display conditions.
On arrival participants will be randomly allocated to one of the three conditions and given a brief on the tasks they will be performing.Once the participant has spent some time acclimatising their vision to the room they will be asked to wear the stereoscopic glasses and then they will proceed to complete a WRRT based reading task in their given condition.All participants will undertake the same reading task presented using a fixed stereoscopic depth effect.
Once the task is completed the user will be asked to complete the visual discomfort survey in a rest area.Each session will be completed in approximately 20 minutes.

CONCLUSION
While there are many open issues when considering touch interaction with stereoscopic displays here we concentrate on limitations introduced by current stereoscopic display technologies.As the use of stereoscopic display technologies expands beyond the scope of the current generation of commercial displays for household viewing, work by the human-computer interaction community will become increasingly important.
In this work we highlight a number of issues that have been raised in the extant literature associated with the use of stereoscopic displays with a particular focus on known negative physiological effect.We hypothesise on how the technological constraints of current stereoscopic displays and their image separation techniques will impact upon users when used in novel interaction contexts.

Figure 1 :
Figure 1: Single display stereoscopic systems use image multiplexing and image separation to present different images to each eye

Figure 2 :
Figure 2: Passive and active filter lenses

Figure 3 :
Figure 3: Appearance of crosstalk resulting from degraded separation of multiplexed images

Figure 4 :
Figure 4: Co-located touch tables promote multiple viewing angles