Haptic User Experience Evaluation for Virtual Reality

Hapticians (engineers, researchers or designers) are developing 'haptic displays' to replicate the complexity of sensations and interactivity accommodated by the hand. Haptic displays hold potential in allowing users to interact with each other and manipulate things in virtual reality (VR), providing them with a limitless safe environment. However, technological advances alone are insufficient to develop a high-quality haptic user experience (UX). Research indicates that current evaluation instruments, guidelines and education on haptic engineering and perception do not address haptic interaction and experience design needs. This research investigates the problem, endeavouring to consolidate findings into helpful haptic UX evaluation instruments, design principles, and guidelines to support haptic design practice.


INTRODUCTION
Progress in real-time gesture recognition and wireless tracking has expanded the potential for haptic applications by allowing accurate interaction and free movement (Zhang et al., 2020). Although few people have used wearable haptics, so the experience they produce is largely unknown. Also the development of haptics is challenging, requiring hardware, firmware and software engineering, alongside interaction and UX design (Iacob and Popescu, 2019). Despite these challenges, increasing numbers of haptic devices are being developed (Giri et al., 2021).
Research shows that hapticians encounter challenges when evaluating haptic UX due to a lack of design tools, guidelines and training afforded by more mature disciplines like software engineering and web design (Seifi et al., 2020). Ultimately technical advances alone are insufficient to ensure haptic technology adoption, making haptic UX evaluation critical to its success. Therefore, this research aims to develop a UX framework to evaluate the quality of experiences produced by haptic devices, such as haptic gloves and handheld controllers commonly used in VR.

(i)
How can hapticians evaluate the quality of UX produced by haptic devices?
(a) What approaches are employed to conduct evaluations with people using haptics? (b) How do hapticians view UX as a response to evaluating haptic technology? (c) What dimensions and factors make up the haptic UX?

HAPTICS: AN INTEGRATIVE REVIEW
Haptic studies include a broad spectrum of topics and methodological solutions, so a systematic integrative review was adopted to investigate the current state of haptic evaluation (Sutton et al., 2019). The review screened 1,702 articles published between 2018 and 2020, excluding 1,634 articles which did not involve humans or failed to meet the inclusion criteria. The final review included 68 articles, which described 76 studies that met the inclusion criteria by employing a wearable or handheld haptic device to investigate or evaluate the human experience with VR.

Results of the review
The final review analysed 76 studies involving 87 experiments which evaluated haptics with people in VR, finding that 96 per cent of haptic studies evaluated physical factors (Figure 1). In contrast, 78 per cent of studies evaluated only a limited range of experiential qualities (Figure 2), and performance evaluations outnumbered experiential evaluations at a ratio of two to one.
Seventy-two per cent of the evaluations took purely quantitative approaches, meaning only 28 per cent of evaluations used qualitative methods to help explain results or understand the user's perspective. Also, only 26 studies defined target users and fewer studies, 11 recruited them.

Figure 1: Word cloud illustrating the range and frequency of haptic factors evaluated by word size.
The review identified a taxonomy of 71 evaluation methods. However, none of these methods was designed to evaluate haptics. The review did not explain the challenges hapticians encounter or why UX methods are not more commonly employed, highlighting the need for further research.

HAPTICIANS: AN ONLINE SURVEY
An online survey is currently underway to answer these questions, comprising a self-efficacy questionnaire and semi-structured interviews to engage with hapticians directly about UX evaluation. Self-efficacy is a person's belief about their capability to successfully achieve a goal or task within a given domain, in this case, haptic UX evaluation (Bandura, 2006). The questionnaire has three sections:  Demographics -to describe the population  Self-efficacy scales -to identify capabilities regarding common evaluation activities  Evaluation questions -to explore knowledge, attitudes, and critical factors

RESULTS OF THE PILOT SURVEY
The preliminary results for the pilot survey questionnaire of 620 randomly sampled hapticians indicate that some issues hapticians experience during evaluations might relate to a lack of engagement with end-users or understanding of the usage contexts. The follow-up interviews with selfselected hapticians revealed the diverse nature of haptic applications. For example, haptic feedback is critical to the teleoperation of nuclear fuel rods, training astronauts to manipulate robotic repair equipment in zero gravity and communicating with the deafblind.

NEXT STEPS
The intention is to develop and demonstrate an evaluation framework based on the haptic taxonomy data, methods and factors identified. The second goal is to describe the challenges and limitations encountered by hapticians during the evaluation process. The final goal is to develop a conceptual model outlining the dimensions of haptic UX with principles to support haptic design practice in developing a more effective and consistent UX.