UX-Handle: A Cloud-based System to Streamline Usability Testing Analytics

User researchers have a poor user experience when analysing data collected from usability tests. Post-usability test analytics is a non-standard process where there is a lack of best practice guidelines and data management workflows. Moreover, usability analytics is normally a tedious and time consuming task. This paper presents a semi-automatic web-based system to improve the user experience of analysing user videos and other data from usability tests. The system presented is called UX-Handle (www.uxhandle.com) and was developed using a web technology stack. UX-Handle allows researchers to input UX-related data including demographic data, systematic usability scale scores, single ease question scores and to efficiently annotate user videos with ‘use’ errors, task completion times and user emotions amongst others. As a result, standard usability metrics are automatically presented using charts and graphs.


INTRODUCTION
Users 'expect' digital technologies to be intuitive.The usability of an interface is a factor in product adoption.Optimizing the user experience (UX) of a digital technology can reduce both the time of task completion and the number of user errors whilst increasing user satisfaction.Optimal UX is when a user is mitigated from unnecessarily 'thinking hard' [1].Demand for usable digital products has driven the adoption of UX practices.The 'usability test' is regarded as one of most effective methods for measuring usability [2].A usability test involves recruiting subjects to undertake a series of tasks [3].During this interaction, the researcher identifies 'use' errors as well as other data.With the emerging theme of data-informed design [2], usability testing has become more summative whilst still being a powerful formative tool.This paper presents a model used in our UX-Lab (www.ux-lab.org)for evaluating products and we present a web-based tool for optimising the workflow of post-usability test analytics.There is a lack of best practices in data management and standardisation of workflows in post-usability test analytics.Each user researcher has their own workflow.This process is not a good UX for UX researchers and is time-consuming.The data structures used by different researchers cannot be easily used in a wider system for benchmarking.Also the continuous switching between spreadsheets and video players can be cumbersome.We developed a solution comprising of a web-based system and a data model to streamline the analytics phase of usability testing.

UX-LAB MODEL FOR USABILITY TESTING
We use a typical usability testing protocol (outlined in [4]).Each task is provided verbally and in print and the user is asked to rate how difficult they expect the task to be.After thinking-aloud during a task attempt, they are then asked 'how difficult the task was?' (this is a Single Ease Question or SEQ [5]).A post-test instrument is also completed (e.g.Systematic Usability Scale [SUS] survey [6]).The UX researcher then analyses user videos whilst deriving metrics such as task completion rates, task completion times, times-until-failure, frequency and severity of use errors, emotion/affective annotations, difference between pre-and post task SEQ ratings, expert vs. novice task completion times and SUS scores benchmarked against a known distribution.

UX-HANDLE WORKFLOW
This paper introduces a system called 'UX-Handle' (www.uxhandle.com)to improve the workflow in usability testing analytics [7].See Figure 1.The analytics workflow consists of three stages, 1) inputting the experiment details and the tasks, 2) inputting user data, uploading user videos and annotating videos with usability events and 3) generation of the usability report comprising of metrics and plots.There are four annotation buttons, 1) task annotation, 2) use error, 3) emotion annotation and 4) research prompts.When an annotation button is clicked, the video pauses and whilst the timestamp is stored in memory, a dialog box appears for the researcher to input metadata.For example, the affective annotation will ask the UX researcher to choose an emotion and the intensity/strength of that emotion using a 10-point rating scale.The video will start playing once that annotation has been completed.Usability metrics and analysis is then automatically generated.

UX-HANDLE IMPLEMENTATION
UX-Handle was developed using a web technology stack (HTML5, CSS, MySQL & PHP). Figure 2 depicts the relational database comprising of eight tables: 1) experiments, 2) tasks, 3) subjects, 4) task annotations, 5) user error annotations, 6) SEA or single ease answers, 7) affective annotations and 8) research prompts.Each test can have many tasks and many subjects.Also, each subject and task can have many events and annotations.

CONCLUSION
This paper presents UX-Handle, a system and workflow to improve the UX of usability analytics.This complements our web-based audit tool to digitise heuristic usability evaluations [8].

Figure 1 :
Figure 1: UX-Handle workflow for post-usability test analytics: Stage 1) inputting experimental details and user tasks, Stage 2) inputting user data and video annotations, and Stage 3) auto-generation of usability metrics, charts and plots.

Figure 2 :
Figure 2: Data model for UX-Handle comprising of entity relationships between the eight tables (point=timestamp).