University of Birmingham Towards the definition of a modelling framework for meaningful Human-IoT Interactions

A social-like environment for a meaningful Human-IoT collaboration has been previously analysed from the perspective of conveying common goals amongst network participants. However, a clearly defined methodology for their implementation of physical systems is required. This work-in-progress paper provides an exploration of the TAFEI framework as a tool for the design and implementation of smart objects from a usability standpoint, in which a detailed description of a system and its user’s interactions could be used by the system designer to architect better interfaces and interaction experiences, focusing on usability and opportunities for ‘intelligent’ IoT behaviour.

•Users may freely distribute the URL that is used to identify this publication.
•Users may download and/or print one copy of the publication from the University of Birmingham research portal for the purpose of private study or non-commercial research.
•User may use extracts from the document in line with the concept of 'fair dealing' under the Copyright, Designs and Patents Act 1988 (?) •Users may not further distribute the material nor use it for the purposes of commercial gain.
Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document.
When citing, please reference the published version.Take down policy While the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has been uploaded in error or has been deemed to be commercially or otherwise sensitive.
If you believe that this is the case for this document, please contact UBIRA@lists.bham.ac.uk providing details and we will remove access to the work immediately and investigate.Download date: 14.Oct. 2023

INTRODUCTION
In recent years Internet of Things (IoT) devices have shifted from passive data collection objects, to things that actively engage with their entities, possessing enhanced capabilities that imbue them with varying degrees of agency and intelligence (Fortino, 2016).
Humans often collaborate with the IoT both as beneficiary of their actions (Guo et al., 2011), and, in some form, as part of their control loops (Schirner et al., 2013).These interactions have resulted in a variety of approaches in which the IoT and its stakeholders organise and interact with each other.
With the objective to fill the gap in the design of 'smart objects' for the IoT that promote meaningful Human-IoT collaboration, this Work-in-Progress paper presents an exploration towards the development of a system modelling framework for the IoT.

BACKGROUND
In Human-IoT interaction, participants often create collaborative partnerships to achieve common goals in social-like structures (Nunes et al., 2015).(Cervantes-Solis et al., 2015) analysed how 'smart objects' and humans collaborate towards common aims, emphasising goal achievement, over system's rules, highlighting the overarching purpose of the system.For human users, the anticipation of an IoT system would be to obtain insight on the devices' operation, added convenience and comfort, richer user experiences or analytics.This expectation presumes a level of trust and common interests, much like social relationships, and as such, Human-IoT interactions have been analysed from the perspective of sociallike organisational structures (Atzori et al., 2014) , where two-way interactions occur, akin to a conversation amongst social actors (Cila et al., 2017).This research focuses on providing an IoT systemmodelling framework through which, by identifying and characterising the purpose of these conversations, meaningful Human-IoT interactions could be attained.
Although modelling for the IoT can be approached from different perspectives, from an usability point of view, (Matassa & Simeoni, 2015) argues that when dealing with "objects whose natural functions" have been augmented by the IoT, affordances "cannot immediately communicate to people their actual values and meanings".This can potentially create a detachment between the system's purpose and the user's actions and goals.
Thus, this work in progress paper aims to fill the gap in this disconnection, by analysing how tasks and goals are achieved in an instrumented object.
Hierarchical Task Analysis (HTA) (Stanton, 2006) and Task Analysis for Error Identification (TAFEI) (Baber & Stanton, 1994, 1996) are tools developed to analyse and design for appropriate system usability through possible system actions available to users.Moreover, TAFEI's modelling methodology with the system's goal at its core, provides a suitable approach to analyse a system's usability, when it has been suggested that its precisely the system's goals that motivates the

Towards the definition of a modelling framework for meaningful Human-IoT Interactions
Cervantes-Solis • Baber 2 interaction, in the form of an overarching theme in this social-like collaboration between human and device (Cervantes-Solis & Baber, 2016).Although TAFEI focuses on analysis for user error prediction, the technique provides a suitable approach to analyse novel forms of interactions on instrumented objects that could become different, but valid, goals not originally conceived in the system's design

METHODS
This study builds upon concepts developed in a previous experiment where an office's coffee making related appliances were instrumented in order to analyse how their interactions aggregated towards a common goal.It developed the notions of topics and themes as the conductive threads in the Human-IoT collaborative environment (Cervantes-Solis & Baber, 2016).
Modelling of meaningful interactions required analysing the system as a network with transitioning states, with each state describing the likelihood of the being used for its original purpose, or if they describe an unexpected goal.As such, the TAFEI framework for modelling was used as a starting point for characterizing how the system's usability could lead to errors.
The experiment presented in (Cervantes-Solis & Baber, 2016) focused on a test bed implemented with limited instrumentation, but supported by the objects affordances, providing information detailing solely to when the corresponding device became active..The study's goal was to show how all devices participated towards a collective goal.
This paper focuses on the model for a single object in the network, exploring the potential deviations in operation that the instrumented appliance can take, and how they could impact the identification of topics and themes in the network.By providing higher granularity in an object's instrumentation, not only a higher detail of interactions could be identified as being part of a topic or theme, but would also provide a means of describing additional topics and themes not previously considered in the original system's design.

A worked example
A coffee machine was instrumented with sensors to detect button presses, displacement of the empty coffee capsules canister and water tank, its temperature, and operation of the handle to place coffee capsules for coffee preparation.
Each instrumented part of the object presents a point of user interaction, and for TAFEI, represents the system image for analysis.
The TAFEI methodology requires the definition of specific user goals, with the intention of constraining the number of possible states.As mentioned, the notion of an association between user and IoT system is the focus of this study, as such, TAFEI's goals are interpreted in this paper as the theme of the collaboration.Hence, the theme defined for this coffee machine is: • THEME_1: Make a cup of coffee Following decomposition of the themes in terms of the states the object's passes to complete it, and the definition of plans through HTA, Table 1 shows a transition matrix for the analysed system, derived from the state and HTA diagrams.Legal transitions (marked as L) for the 'coffee making' theme are shown in the main diagonal.TAFEI is generally used to find possible errors in product usability design.However, in the case of an instrumented device with the potential of being imbued with a degree of intelligence, such as the one presented in this paper, it becomes useful to identify other interactions that could enable different goals or themes that could provide useful information to the user.The transition matrix shows the valid path towards the specified theme, whilst also providing information on the illegal (I) transitions in the context of said goal, as well as those that are impossible (shown in table 1 as '-') As such, another theme could be defined: • THEME_2: Coffee machine servicing.
Table 1.Transition matrix for ´coffee making´ theme.Highlighted in green is the path leading for the successful completion of the goal.In yellow, although invalid for the analysed theme, an emergent secondary theme is enabled, showing the required path for ´servicing´ the coffee machine.

DISCUSSION
TAFEI was originally conceived as a goal based modelling framework, allowing identification of invalid actions or goals that possibly were necessarily not part of the original design.Hence, it supports a product's design and implementation process to consider circumstances in which unexpected object or user behaviour could hinder or modify its usability.
For the IoT, TAFEI is repurposed to model ways in which the system's functionality could be extended.In contrast to the system used in the experiment that motivated this research, in which objects were only instrumented to detect whether they were used or not, higher granularity in sensor instrumentation allowed a thorough exploration of a device's states and transitions.Hence, it would be possible identify other topics (sub-goals or actions) and thus, present additional knowledge to the user.For example, in the described scenario, the coffee making theme for the coffee machine (it's main goal) is clear, with precisely defined states, plans and transitions.
From a device perspective, in a scenario of a 'dumb' coffee machine, the definition of all possible themes enabled by the system might not be relevant, as it will only carry on with its expected functionality without actually extending its capabilities.However, for a 'smart' machine, the extension of 'servicing' theme could be used to allow the machine to pre-emptively notify the user whether the machine needs descaling (by keeping track of the number of times the water has been replaced and the amount of coffees made), or that the empty capsules canister needs emptying.This implication would in fact enable the degree of intelligence pursued by most IoT ecosystems.
From an usability standpoint, a more detailed description of a system and its user's interactions could be used by the system designer to architect better interfaces and interaction experiences, focusing on usability and opportunities for 'intelligent' IoT behaviour.
As a work-in-progress, further development needs to be accomplished; mainly to address the issue of how the system would be able to identify which goal is which when more than one is present.Fully automated decision making nodes need to be implemented to evaluate when and which notifications could be made to the user, taking into consideration results and data from previous machine learning efforts.
Supplementary activities need to be carried out towards establishing a clearly defined framework, and to test and measure its capacity to properly describe and define a meaningful Human-IoT collaboration in a social-like environment, capable of managing its themes.