Exploring Next-Generation Touch-Rich Interactions for Consumer Well-Being

This article critically examines the touchscreen as a lived technology and argues that next-generation devices should take advantage of more ‘touch-rich’ interactions. We begin from the premise that the ubiquity of touch-screen devices, combined with the frequency of use, appear to have a negative effect on consumer well-being (Lee et al. 2014; Elhai et al. 2016). Today’s industry heavily relies on the image-based economy (of photographic desires, concepts, and visual attention), which significantly contributes to the information overload, unhealthy consumption of visual information through basic touch interactions and largely overlooks designing for other senses (Schroeder 2004; Shedroff 1999). The smoothness of the touchscreen provides an ideal interface for an unbroken visual information stream; this is its benefit from an industry standpoint but is also linked to several concerns regarding consumer well-being, due to an intense, constant influx of informational ‘noise’ (Himma 2007). We argue that the infinite stream of incoming information could be reduced by shifting focus to developing touch-rich interactions for tactile senses instead of visual-based perception, and suggest this as a clear, near- future direction for interface design, whereby touch replaces noise. Combining insights from interaction design, product design and cognitive psychology, we argue for more touch-rich interface experiences as a mode of disrupting current device conventions. We project five keyways in which touch-richness might enhance user experience, with implications for consumer well-being. Touch. Tactility. Touch-poor interactions. Touch-rich interactions. Design for touch-rich experience.


INTRODUCTION
The proliferation of the touchscreen interface has in the last two decades significantly shaped patterns of technology use: swiping and scrolling, the unending newsfeed, design of hardware and software that enable ever-closer versions of the 'frictionless' interface. The seamlessness of the capacitive touchscreen can be seen as a phase in a broader trajectory in which our technologies move from visible and mechanical interfaces, toward ultimately invisible and embedded interfaces. This evolution involves, for example, the shift from button-based interaction and manipulation of physical parts to gestural movements and haptic interactions (with overlap between these) (Srinivasan & Casdogan 1997;Saffer 2008).
In the last ten years, the ability to produce hardware and software interfaces approaching the ideal of invisible and embedded has become easier and thus has accelerated. Simultaneously the cost of developing and integrating touchscreens has declined, meaning that now almost any surface can host one (White et al. 2017). Not surprisingly, smart surfaces are prime real estate for advertisers and content producers, with newsfeed apps making these spaces ever more 'sticky' via endless scrolling interfaces that deliver a high volume of 'content' extremely efficiently. This is excellent news for those who wish to keep consumers online and engaging for longer periods of time, and indeed the time spent on screen devices has increased steadily, and across a period of years coterminous with the rising popularity of touchscreen interfaces.
Perhaps not surprisingly, so much time spent staring at and interacting with touchscreens has negative effects on the well-being of consumers, where well-being is understood as being commensurate with 'good health' and comprising measures of physical and mental health related to human thriving (Breslow 1972). There is widening range of research suggesting that touchscreen features have detrimental effects on many users, where frequent use of touch screen devices has been linked to compulsive engagement behaviours (endless scrolling) that has similar qualities to addiction (Weiderhold 2018), a related disruption to self-regulation (Coyne et al. 2019), and anxiety and depressive symptoms (Elhai et al. 2016). More time in noisy smart-screen environments is also shown to have a negative impact on users' cognition, manifesting as shifts in cognitive processing, dramatically diminished attention spans, delayed or deferred social skill development, and decreases in fundamental literacy skills (Ward, Duke, Gneezy & Bos 2017;Microsoft Canada 2015;Bauerlein 2008;Carr 2011;Turkle, 2011;Small & Vorgan 2008). A macro-effect is the high visual-verbal informational load, which has had a demonstrably huge effect on user experience. It's clear that our relationships with our devices are prevalent, intense, and highly consequential for well-being.
The seamless, endless, integrated, almost-invisible nature of the touchscreen is in many ways its most dangerous attribute. The screen interface is what we would describe as touch-poor: it is without any form of textural richness or sense-based legibility, and neglects the need for touch, or more specifically, the need for rich haptic input. Endless interactions with the texturally-blank touchscreen is the haptic equivalent of staring at a blank wall.
Touch is central for many dimensions of well-being, and this is directly related to its informational functions: hands, particularly fingers, have an extremely dense presence of nerve endings, making them one of the richest sources of tactile information we have, involved in knowledge production and sense experience even before birth. There is, not surprisingly, a large amount of evidence demonstrating how touch experience and well-being are linked. The 'need for touch' has been identified in marketing and consumer behaviour, the phrase used to describe a common trait where haptic information is highly valued and influential in decision-making (Peck & Childers 2003;Yazdanparast & Spears, 2012). Other fields bear out the 'need for touch' in terms of measurable psychological benefits: touching other humans reduces stress, touching inanimate objects and textures produces feelings of pleasure; touching natural materials increases positive emotion (see for example Feldman et al. 2010;Bhatta et al. 2017). Tactile experience can even enhance mindfulness, stabilizing and broadening the scope of attention (Stanko-Kaczmarek & Kaczmarek 2016). Yet thus far the role of touchsensory experience in device interaction seems to have been relegated mainly to a secondary or tertiary position in design considerations. That is, touch affordances of interfaces have broadly been treated as having a supportive role in the process of accessing (textual/visual) information, rather than touch being understood more accurately as an informational process in itself (Hartson 2003;Serrano, Banos & Botella 2016).
Ours is one of several recent studies shifting away from a view of frictionless and increasingly imperceptible interfaces as the ideal design standard, toward interface designs that involve more, and more meaningful, forms of textural richness and sensual legibility. This is with the assumption that, within a trajectory of ever more integrated, invisible and efficient interfaces, the negative effects of high-volume, high-speed visualverbal informational load will continue to intensify, unless there is intervention at the level of design and ideally of policy. In this paper we propose design interventions, arguing for a conscious directional shift in design principles that relate to interface texture and haptics. With the view that touch is an important informational process and that touch-rich experiences contribute to well-being, we identify specific user needs related to well-being that are not being fulfilled by current smart-screen interfaces, and we speculate on future directions for interface design that engage the haptic sense in better ways. We introduce the concept of touchrichness (TR) as a conceptual and practical design solution to many of the challenges generated by touch experience of smart screen technology.

TOUCH-POOR INTERACTION DESIGN
In this paper, we define touch-poor interactions (or touch-poor user experience) as lacking textural richness or sense-based legibility, and neglecting the need for rich haptic input. While 'frictionlessness' has become an accepted design standard, it is also the result of strategic interface design choices that optimise for content volume, consumer engagement in the form of clicks and views, and cultivating consumer compulsion to stay within a proprietary environment (for example a particular app) as long as possible. The outcome of design decisions motivated in this way are screenbased experiences characterised by high visualverbal informational load (noisy, scrolling). To facilitate these experiences, haptic information is minimised, with touch interfaces broadly designed to be smooth, texture-less and physically rigid, which avoids disrupting the consumer's absorption in media content.
It has been established that humans have developed a daily 'need for touch' and a lack of fulfilment of tactile needs is also associated with lesser well-being (Patrick et al.;2007;Peck, Wiggins & Johnson 2011). Recent research suggests that tactile need unfulfillment is related to increased smartphone use, whereby consumers with 'high in the need for touch may demonstrate an overuse of a smartphone's touch screen to satisfy this need' (Lee et al 2014). Or to put it differently, the lack of haptic information leads the consumer to seek stimulation through visual-verbal input, intensifying the cycle of compulsive consumption (Elhai et al. 2016). This finding suggests that the touchscreen itself serves as a digital surrogate for human tactile need fulfilment. However, the need is not meaningfully fulfilled by the device, with the glassy smoothness of the mass-produced touchscreen providing only minimal sensory experience. Diversity of type, scale and material is crucially important to sensory processing, where the function of touch is of course not evolved to manipulate a perfectly smooth interface, but rather to produce diverse knowledge about physical relationships and mechanics in the world in a variety of circumstances (Lederman & Klatzky 1987).
In concrete terms, general textures such as roughness, hardness, softness, flexibility, grippiness, etc. are necessary for the haptic sense to function normally. However, these dimensions are not often nor actively incorporated into interface design for wide distribution, and they do not fit the sleek design standard that has become normal through the proliferation of mass-produced touchscreens, an aesthetic propagated by firms such as Apple. Ironically the sleek design is associated with quality and value, and yet the portability, size, shape and minimalist design conveniently and seamlessly support commercial services designed to exploit consumer resource, for example maximising time spent online, directing users to take certain actions, or encouragement of in-app purchases. These are about maximising corporate value chains, rather than representing genuine value for consumers (for whom well-being is of considerable value).

TOUCH-RICHNESS AS AN ALTERNATIVE DESIGN STRATEGY
Touch-rich describes hardware interface design which is rich in haptic information. Our aim in developing the TR concept is, put simply, to provide a framework that can enable designers and technology firms to develop better interfaces: better for individual quality of life, and better for broader public health and well-being. TR includes four design considerations which are also dimensions of what we're referring to as 'richness': texture, elasticity, gesturality, and interpolation. These dimensions interact to produce touch-rich experiences. The function of conceptualizing TR in this way is to provide a clear basis for interaction and interface designers to direct solutions in specific ways.
In the following sections we analyse established and emerging interface design concepts and prototype through a TR lens, highlighting those we find promising from a well-being perspective and provocative as design concepts. Importantly, these four dimensions of TR involve the relationship between visual-verbal informational load and haptic informational load within user experiences, where the haptic and the visual-verbal tend to be inversely related. Importantly, informational load refers to the volume of information being processed, while richness refers to the type of information being processed, where diversity of type has greater richness. The dimensions identified are: Texturality: The degree to which touch experiences are diverse in feel, which enriches the haptic informational experience. For example, interfaces that are soft, ridged, have raised or etched surfaces, or unexpected or uncommon haptic feedback. An example of textural TR is the Sony Xperia Touch Projector (see Figure 1) which turns any flat surface into an interactive touchscreen (2017). This product is touch-rich in texture because it exposes users to various surfaces rather than just screens, and enriches their tactile experience, when interacting with the interfaces. A speculative example of textural TR is a 'Skin-on Interfaces' project (see Figure 2) which illustrates how artificial skin for mobiles phones provide warmer interaction and input (Teyssier 2019). Skinlike interfaces allow users to interact with the more sensitive and natural textural intention. Elasticity: Touch experiences which are flexible, mutable, and organic. For example, interfaces that respond to physical touch by changing shape, feel, or temperature, thus providing a greater variety of haptic information through interaction or over time.
The recent series of foldable Samsung Galaxy Z Flip 3 5G and Fold 3 5G phones are an example of elastic design TR (2021). Samsung Galaxy Z Flip 3 5G phone (see Figure 3) is a smartphone which can be flipped and folded like a compact mirror and Samsung Galaxy Z Fold 3 5G phone (see Figure 4) Gesturality: The extent to which haptic experience breaks the link between direct physical contact and command. For example, swiping quickly left or right is more gestural than the drag-and-drop, requiring less touch to execute the command. Gesturality is associated with less haptic information, as there is no physical contact necessarily. For example, the Magic Leap 1 virtual headset (see Figure 6) allows users to interact with augmented reality (2020). It has accurate hand tracking technology that enables users to use hand gestures such as pointing, shaking, fist bumps and tennis serves which shows the application of high gesturality in TR. HaptX is the award-winning company which developed HaptX Gloves DK2 (see Figure 7) (2021). This technology enables the user to experience realistic touch such as heat, cold, and haptic realism in virtual reality. Virtual reality and augmented reality technologies and their accessories produce opportunities for designing for more haptic experiences in virtual reality that are touch-rich in gesturality. Interpolation: The extent to which physical touch disrupts visual information flow on a device. For example, fast-scrolling functions enabled by a quick swipe are low interpolation (very little disruption of informational flow). Conversely, interfaces that require interruption of visual information flow-for example, the 4G mobile phone Punkt MP02 Pigeon (see Figure 8) has simple, tactile and grippy hardware that encourages the user to interact with the device via a tactile keyboard (Punkt 2021). This is higher interpolation, because the user is compelled to stop processing visual-verbal information to shift to tactile information processing.
Similarly, the Sidekicks project (see Figure 9) is a series of speculative objects such as an alarm clock, lamp, projector, and speaker, which can only function if the user's smartphone is placed into them (Bandi 2018). The objects are controlled with red tactile buttons and each object ceases to function if the user attempts to interact with the smartphone. This example also shows high level of interpolation in TR.

CHALLENGES TO TOUCH-RICH DESIGN
Although the need for better interface design is clear, and ways in which that can be achieved through TR can be productively applied in many cases, there are limitations respective of technological and societal, as well, as market, constraints. Digital technology favours cheap, ubiquitous duplication, and results in products and patterns of use characterised by abundance, modularity, and recombinatorial properties (fungibility). While elements of this will doubtless be embedded in different kinds of next-generation haptic interfaces, it will be, at least initially, less likely that unique hardware interface components will fit that model seamlessly, nor is it clear that they should. This paper takes the view that, whilst there are clear and obvious barriers to introducing design principles that do not mainly and specifically serve the commercial interests of technology firms according to the current status of device use and commercial norms, there is a significant value proposition in designing to these considerations with respect to longer-term commercial and civic viability, for example changes in market trends related to mental health and well-being, innovation in experience design, and policy change concerning commercial responsibility in relation to mental health.
It's clear that the existing standard is not conducive to well-being, considering wide indicators of mental and physical health associated with overuse of devices optimized for visual-verbal information. It is interesting to consider what kinds of device interfaces might arise if there was greater attention to haptic richness by interface designers and companies both, including the importance of touchrichness to experience design and the further development of specific design methods aimed at achieving it. In practical terms, developing new hardware design is likely to involve using existing materials differently, and sourcing new materials to develop new functions (for example, shapechanging and colour-changing polymers, smart fabrics and e-textiles, soft circuit technologies, and graphene-based conductive elements) (Nabil et al. 2017). These processes can be costly to develop and may require materials that are costly to use in production. Speculative concepts and prototypes that may never become commercial products are useful in this context, to help work through the design and production challenges, toward a middle ground: realising some of the ambitions of wellbeing focused hardware and software interfaces, mitigating negative effects of touch-poor interactions, and developing interfaces that may even have a positive impact on well-being.
Another main challenge faced by many design researchers is measuring the benefits and impact of better design on well-being for consumers. In this case, it would be measuring whether touch-rich design is a better solution for users. Pakalkaitė explores the practical application of touch-richness by designing a speculative prototype called Shapie (2022). She argues that the use of design fiction and sparking the debate of the touch-rich prototype can address the potential misuses of the technology and negative impact on well-being (Sterling 2005;Dune, Raby 2014). This study is in its infancy, and touch-richness research is currently limited by the lack of data and further explorations are needed to confirm and measure whether it is a better solution to design for wellbeing.
While it's not straightforward to predict outcomes, it's easy to imagine that reducing the vast cognitive load placed on the average smartphone user's visual-verbal faculties for hours every day and redistributing this load to be shared across the senses in gentler, more diverse, and possibly more interesting ways, would have benefits to quality of life. This is likely to have even more relevance for those with forms of neurodivergence such as ADHD, where rates of addictive behaviours related to smartphones may be more pronounced (Kwon et al., 2021). Broadly speaking, less contact with the monoculture of newsfeed doomscrolling would feasibly reduce urgency and stress for many if not most mobile device users. And even more broadly, it seems clear that having relationships with objects, such as phones, that are based on need fulfilment and pleasure, rather than addictive or compulsive attachment, would be healthier for consumers as well.
There is scope to increase the diversity of hardware interfaces, where the gravitational norm is increasingly the generic, minimalist, flat, smooth rectangle. Aside from considerations already discussed, this minimalist design described isn't a particularly inclusive one for anyone with visual impairment, literacy challenges, or upper extremity disabilities (Mi et al. 2014). Focusing on non-visualverbal modes of information experience design are likely to provide solutions to this. Furthermore, the nature of smart interface design evolution is such that those who own smart devices with touchscreens are enculturated to a form of interface-driven (and fairly abstract) interaction norms that aren't intuitive to someone holding a smartphone for the first time but become endemic in culture due to the popularity of these devices. The lack of diversity in design leads to a generic culture of interaction, and one that may exclude those who are unable to afford expensive devices, as well as those who may not be digitally-native, to whom the devices or their habits of use may feel alien and excluding. Ultimately, a diverse public is better served by a diverse suite of design possibilities and philosophies. To the extent that the commercial potentials of touchscreens encourage all surfaces to become homes for touchscreens, the current pattern does not bode well for consumers who wish to have richer experiences and better lives.

IMPLICATIONS
The inclusion of speculative designs here has hopefully helped to illustrate the extent to which TR is future-focused. If TR dimensions form a series of design provocations, it's interesting and worthwhile to consider the outcomes of these provocations as they might emerge across specialized areas of interface design, such as smartphones. The implications of a normalized touch-rich experience of a smartphone, wherein rich haptic experience is integrated into the functionality of the device, might be a completely different kind of smart phone: one that changes shape in response to body position, changes texture according to who you're speaking with, measures your heart-rate so it can adapt sound and colour to reduce your stress levels, uses facial recognition technology to respond to your expressions, growing warm when you smile for example-and so on. This is not just about how we interact with devices of course, but about how they interact with us, our relationships to our objects and how we communicate through them and with them. The conscious design of touch-rich communication interfaces also has potential to change some of the fundamental coordinates of how we communicate in general. TR exists within a paradigm of care and proposing TR design assumes that comfort and intimacy are not just to be delivered in parallel to communication outcomes, but that for the sake of well-being they are necessary components of communication.
If TR was to become more widely explored by technology firms, its outcomes are likely to be interventions, processes, products and services that provide new forms of care embedded in new technologies. As such, outcomes would be complex in relation to the ethics of care-focused design, both in terms of how experiences of care are made commercial, and the ways in which forms of intimacy with devices will engender altogether new benefits, as well as new forms of risk, for human-focused interaction design. There are also complexities in balancing dimensions of TR design even in the examples given above, where responsive design features may for example work against interpolation. That is to say, change is not simple, and shifts in direction will be attended by complex discussions of the implications of every part of every new device that becomes widely adopted.
There are implications for government policy relevant to this area which we can already expect, concerning the responsibilities of commercial organizations to administer well-being as part of consumer product design. In time there could be specific conversations around the dangers of overreliance on devices that are too effective. For example, if human-device interactions could positively shift our moods, feelings or responses without much effort, this would no doubt be interrogated as inhibiting normal emotional development, or decreasing a user's ability to selfregulate-bringing us back to the kinds of problems that we have with current technologies. More broadly however, there is an opportunity to interrupt the current trajectory toward frictionless interaction design, whose outcomes have been broadly determined according to a logic of high-volume consumption, and to design with different interests in mind: the well-being of those who use the devices.