Heuristics for Evaluating Multi-Touch Gestures in Mobile Applications

Multi-touch gesture interaction is one of the main features that differentiate the current smart mobile era from the desktop-computing era. Heuristic evaluation is a popular usability evaluation method, mainly due to being less time-consuming, more cost effective, and applicable to different stages of the design and development phases. Researchers have provided their own sets of heuristics targeting domain specific mobile applications or some specific features (e.g., contextual behaviour). In this work, we specifically target evaluating multi-touch gestures in mobile apps through a set of 15 heuristics, which we selected after analysing previously proposed sets of heuristics. However, we adjusted these selected heuristics to make them appropriate for evaluating multi-touch gestures. We conducted a preliminary study with five evaluators in which they were able to find out more usability problems related to multi-touch gestures in the used mobile app compared to a previously proposed heuristics set for mobile apps. Heuristic evaluation. Usability evaluation. Multi-touch gestures. Mobile apps. Smart mobile devices.


INTRODUCTION
Heuristic evaluation, initially proposed by Nielsen and Molich (1990) and later compiled by Nielsen (1994) to 10 famous heuristics, is an expert-based analytical evaluation method that focuses on finding out usability problems through the guidelines of these heuristics.The main reasons for its popularity are its easy usage and the fast and inexpensive application (Nielsen, 1995), as it can be done within a single-day time frame as well as at various stages of the interface development from early sketches to implemented systems.Nielsen showed in a study (Nielsen, 1994) with 5 evaluators that they were able to find out almost 75% of total usability problems in the interface.
Later, many other researchers provided their own sets of heuristics, e.g.: Shneiderman and Catherine (2004) proposed 8 golden rules for interface design and key points, which should be avoided during design.While Tognazzini (2014) proposed 16 list of design principals, which can be categorized into two heuristic categories (as per according to Atkinson et al. 2007): the defaults category for system configurations and text-based field data, and the learnability category for minimizing and learning curve.
The initial heuristics by Nielsen and the later proposed sets by other researchers mostly focus on user interface design for desktop-computing environments.However, designing for mobile devices (i.e., smartphones and tablets) and mobile applications (mobile apps) brought new challenges compared to the desktop-computing era (Dunlop and Bewster, 2002), e.g., mobility, contextual availability, limited input/output, multi-touch gestures, etc.Hence, previously proposed heuristics may not comprehensively find out usability problems during the evaluation (Bertini et al., 2006;Po et al., 2004).
To tackle this shortcoming, researchers came up with their own sets of heuristics for mobile devices and applications.Some examples for these sets are: Po et al. (2004) proposed two lightweight variants of heuristics: heuristic walkthrough that combines heuristics with scenario of use, and contextual walkthrough that involves conducting heuristics in the field; Bertini et al. (2006) proposed a set of heuristics for capturing mobile contextual requirements; Korhonen and Koivisto (2006) proposed a set of 11 playability heuristics for evaluating mobile games; Neto and Pimentel (2013) proposed a list of 11 heuristics by extending Nielsen heuristics for the usability evaluation of mobile user interfaces; Inostroza et al. (2013) proposed 12 usability lists for touchscreen-based mobile devices; Chuan et al. (2015) targeted heuristics for gestures interaction in general; while Joyce and Lilley (2014) proposed a list of 11 heuristics for smartphone mobile apps.
One of the main features that differentiate the current smartphone era to the previous mobile phone era (i.e., before the launch of first Apple iPhone) and the desktop-computing era is the proper utilization of multi-touch gesture interaction facility, which provides a more natural and intuitive interaction mode.The previously proposed sets of heuristics either target at mobile devices and apps in general (e.g.: Inostroza et al., 2013;Jpyce and Mariana, 2014;Po et al., 2004), or some other specific feature (e.g.: Bertini et al., 2006;Korhonen and Koivisto, 2006), or gestures in general (e.g., Chuan et al. 2015).
In this work, we target evaluating multi-touch gestures in mobile apps through specific heuristics.For this, we analysed previously heuristics and came up with a set of 15 heuristics, where 14 out of them are taken from the previously proposed heuristics by different researchers.However, we adjusted these selected heuristics to make them appropriate for evaluating multi-touch gestures in current mobile apps.We performed a preliminary study with five evaluators, in which they evaluated multi-touch gesture interaction in a mobile app using our proposed heuristics compared to the heuristics set proposed by Joyce and Lilley (2009).Results of this preliminary study indicate that evaluators were able to find out more usability problems related to multi-touch gesture interaction through our proposed heuristics.
In the remainder of the paper, first we provide our proposed set of heuristics and then we describe the conducted evaluation study and the results.Finally, we conclude the paper and shed light on future directions.

HEURISTICS FOR MULTI-TOUCH GESTURES
We analysed previously proposed heuristics and propose a set of 15 heuristics for evaluating multitouch gestures in mobile apps.Out of these, 14 are taken from previously proposed heuristics sets by different researchers and modified to make them appropriate for evaluating multi-touch gestures.Here, we provide our set of heuristics, their sources, and the adjustments towards the multitouch gestures evaluation.(Inostroza et al. 2012;Nielsen, 1995): Gestures other than tapping or sliders are normally not visible to users.The system should visibly, clearly, and concisely provide feedback for multi-touch gestures feedback.

Matching between the system and the real world behaviour (Nielsen, 1995):
A multi-touch gesture in a mobile app should enable the functionality that matches the real world.For example, performing a left-swipe gesture should bring the next object from the right side (i.e., showing the next picture from the right side).(Nielsen, 1995): Many times, users can perform unwanted gestures on the touch screen of smartphones and tablets (e.g., tapping, swipe, etc.), which could bring the system to another state.The system should support undo and redo for a clear navigation.

Consistency and standards (Nielsen, 1995):
A certain gesture should perform the same kind of action throughout the mobile app and a certain action should be performed by the same gesture throughout the app (e.g., pinch-out gesture for zoom-in the map throughout the map navigation app).Further, it is better not to conflict a gesture's operation with the underlying OS standard gestures' operations.

Realistic error management (Chuan, 2014):
Users may not always be precise in performing multi-touch gestures on mobile devices, due to the direct usage of fingers on touch screens.Expecting much precision during gestures interaction leads to mistakes; therefore, the system should allow wide margins in gesture interaction.(Nielsen, 1995): Experienced users may prefer to configure gestures as per their own requirements (e.g., double tapping on a particular area for direct zoom-in a specific part of the map).Therefore, the system should also provide configuration options and shortcuts to manage gestures for expert users.(Nielsen, 1995): Multi-touch gestures provide an intuitive way to interact with mobile touch-screens, which encourages users to explore the underlying interactions.Therefore, the system should avoid providing unnecessary information about the supported gestures on mobile app interfaces.

Help and documentation (Nielsen, 1995):
Multi-touch gestures could be difficult to understand for new users.Therefore tutorials, documentation, and help should be provided to explain the usage of the supported gestures.(Miranda, 2014): How much a certain gesture is enjoyable (e.g., using two fingers rotation gesture to rotate the underlying object like a map or a picture in the app), as well as the avoidance of making negative user experience (e.g., flicking over a photo album but moving the pictures slowly).

Learnability (Chuan, 2015):
The used multitouch gestures should be easy to perform and easy to learn for users (e.g., using two fingers to rotate an underlying object rather than using a complicated gesture).

Cognitive workload (Chuan, 2014):
The system should support gestures with less cognitive workload.Therefore, it should avoid recalling the complicated hard finger configuration gesture that could result in increasing user memory load.
12. Fatigue (Chuan, 2015): Due to involvement of hands, users may get tired after a long usage.Therefore, the system should use as precise multitouch gestures as it could be rather than making them unnecessary longer (e.g., where tap can work then there is no need for double tap).(Nielsen, 1995;Tognazzini, 2014): The user should not need to remember a specific gesture usage from one part to another.It should be obvious to recognize the underlying gesture for interaction with the underlying object.

Do not lie to the user (Bevan, 1997):
A multitouch gesture should perform the action as it is supposed to do in normal circumstances (e.g., swipe left is normally use for moving the right object to the focus part rather than any other activity) and should not refer to some unexpected behaviour.

Screen orientation:
The provided gestures should work on both screen orientations (i.e., vertical and horizontal) as well as on all screen sizes.

THE EVALUATION STUDY
We conducted a preliminary evaluation study in order to check the feasibility of our proposed heuristics for multi-touch gestures.We selected the 11 heuristics for mobile apps by Joyce and Lilley (2014) for comparing with our proposed set of heuristics.The goal of the study was to investigate how many multi-touch gestures related usability problems can be found while using both sets of heuristics.We hypothesized that the evaluators would be able to find out more usability problems related to the provided multi-touch gestures using our set of heuristics compared to the heuristics set previously proposed by Joyce and Lilley.
We performed the study with five participants who acted in the study as heuristic evaluators.These participants were computer science postgraduate (master or PhD) students.The study was done in within-subjects manner, where 3 randomly chosen participants performed the evaluation using our heuristics on day one and then using Joyce and Lilley heuristics on day two while the other two randomly chosen participants performed the evaluation in the reverse order.Before performing the evaluation, we gave a detailed tutorial, lasted from 30 to 60 minutes, to each participant for the corresponding used heuristics set on that day.
We used Dolphin web browser on mobile devices and asked the participated evaluators to work on four scenarios that needed some kind of gestures interaction (e.g., creating user-defined gestures for choice symbol in Dolphin web browser).We used three devices (i.e., OnePlus One with Android, iPad with iOS, and iPhone 6 with iOS), which were assigned randomly for each experiment.The participants were asked to note down the found usability problem against each heuristic after completing each task, as proposed by Nielsen (1994).In the case of any difficulty, participants were allowed to ask the questions.

RESULTS
In this section, we describe results of our conducted user study with five evaluators.Figure 1 shows the result of found usability problems by all five evaluators in all four tasks, while Figure 2 shows the result of found usability problems by all five evaluators in each of the four tasks.From these results, it is clear that participated evaluators were able to find out more usability problems related to multi-touch gestures in all four tasks using our set of heuristics.In the case of Joyce and Lilley heuristics, the participants (who worked as evaluators) were able to find out in total 28 usability problems (i.e., 7 usability problems by Evaluator-1, 4 usability problems by Evaluator-2, 7 usability problems by Evaluator-3, 5 usability problems by Evaluator-4, and 5 usability problems by Evaluator-5) in all four tasks (7 usability problems in Task 1, 10 usability problems in Task 2, 2 usability problems in Task-3, and 9 usability problems in Task 4) using 6 different heuristics.Overall, all the evaluators found six usability problems related to the heuristic "provide immediate notification of application status", seven usability problems related to the heuristic "use a theme and consistent terms, as well as conventions and standards familiar to the user", four usability problems related to the heuristic "prevent errors where possible; assist users should an error occur", six usability problems related to the heuristic "display an overlay pointing out the main features when appropriate or requested", one usability problem related to the heuristic "each interface should focus on one task", three usability problems related to the heuristic "intuitive interfaces make for easier user journeys", and one usability problem related to the heuristic "allow configuration options and shortcuts".
On the other side, they were able to find out in total 43 usability problems (10 usability problems by Evaluator-1, 6 usability problems by Evaluator-2, 11 usability problems by Evaluator-3, 8 usability problems by Evaluator-4, and 8 usability problems by Evaluation-5) in all four tasks (10 usability problems in Task 1, 15 usability problems in Task 2, 14 usability problems in Task-3, and 12 usability problems in Task 4) using 13 heuristics from our proposed set of heuristics.Overall, all the evaluators found six usability problems related to the heuristic "visibility of system status", two usability problems related to the heuristic "matching between the system and the real world behaviour", one usability problem related to the heuristic "navigation and user control", six usability problems related to the heuristic "consistency and standards", two usability problems related to the heuristic "realistic error management", one usability problem related to the heuristic "aesthetic and minimalist design", five usability problems related to the heuristic "help and documentation", one usability problem related to the heuristic "joy of use", six usability problems related to the heuristic "learnability", four usability problems related to the heuristic "cognitive workload", five usability problems related to the heuristic "fatigue", one usability problem related to the heuristic "recognition rather than recall", three usability problems related to the heuristic "do not lie to the user".
At the end of the experiment where the participants used our proposed heuristics, we asked them to rate our heuristics with regard to the usefulness for evaluating multi-touch gestures in mobile apps.The rating was based on Likert-scale with 5 scales, from scale 1 "Strongly Disagree" to scale 5

Participants' Rating
Strongly Disagree Disagree Neutral Agree Strongly Agree "Strongly Agree".Figure 3 shows the rating provided by the five participants.The rating result shows that for the most of heuristics, participants provided either "Strongly Agree" or "Agree" rating.However, we received exceptional feedback in two heuristics.In the case of heuristic number 6 (i.e., allow configuration options and shortcuts), 3 participants gave neutral feedback and 2 gave "Agree" rating.While in the case of heuristic number 15 (i.e., screen orientation), 4 of the participants were "Disagree" about the usefulness of this heuristic and 1 participant was neutral.The reason behind the feedbacks in these two heuristics could be the nature of the underlying tasks; however, detailed evaluation study is needed in order to exclude them from the proposed set.

CONCLUSION AND FUTURE WORK
In the past, few researchers have proposed their heuristics sets to cope with the challenges of the current smart mobile era, which is fundamentally different to the traditional desktop era on many scales.However, only Chuan et al. (2015) partially targeted gestures in general.Due to the importance of multi-touch gestures in current mobile apps, we defined a set of 15 heuristics, partly taken and modified from previously proposed heuristics.We provided our adjustment to these heuristics in order to make them appropriate for evaluating multi-touch gestures in current mobile apps.
In a preliminary evaluation study, five participating evaluators were able to find out more usability problems using our heuristics compared to the heuristics for mobile apps proposed by Joyce and Lilley (2014).Further, they provided high ratings for most of our selected heuristics regarding their usefulness in the evaluation process.However, these are only initial results and cannot be generalized due to some limitations in the study (e.g., using only one mobile app, few number of tasks, etc.).In the future, we intend to perform an extended study with experts from industry in order to check the effectiveness of our heuristics set.Further, we plan to conduct evaluation studies using different mobile apps at different stages of development and with different levels of tasks in order to generalize the results regarding the feasibility of our heuristics set.

Figure 1 :
Figure 1: Total number of usability problems found by each evaluator in all four tasks.

Figure 2 :
Figure 2: Total number of usability problems found by all 5 evaluators, categorized by tasks.

Figure 3 :
Figure 3: Evaluators' rating for the new proposed heuristics about their usefulness during the evaluation process.