Evaluating the Delivery of Assisted Living Lifestyles at Scale (dallas)

This paper presents an overview of the dallas (Delivering Assisted Living Lifestyles at Scale) programme and discusses a preliminary evaluation framework being developed by the University of Glasgow in collaboration with the dallas programme stakeholders. The dallas programme aims to deliver independent and assisted living solutions across communities in the U.K at scale aiming to reach up to 169,000 individuals between 2012 and 2015. The evaluation of the impact of the dallas programme on care systems and individuals will be undertaken using a multidisciplinary, collaborative, robust and pragmatic mixed-method evaluation framework. The evaluation is designed to capture experiences, derive empirical lessons and share actionable knowledge with stakeholders in the deployment and roll-out of assisted and independent living solutions. This paper briefly describe a preliminary evaluation framework and discuss some of the challenges which arise with a study of the nature, complexity and scale of dallas.


INTRODUCTION
The dallas programme (launched in May 2012) is funding four service-led consortia (known as 'communities') throughout the UK : i-Focus, Year Zero, The Feelgood Factory and Living It Up (see Figure 1).These 4 communities have been awarded £25 million funded jointly by the Technology Strategy Board, the National Institute for Health Research, the Scottish Government, Highlands and Islands Enterprise and Scottish Enterprise.The communities have also raised their own financial contributions bringing the total programme funding to £37million 1 (see Figure 1).
The aim of the dallas programme is to demonstrate how assisted, independent living and innovative technologies can be used for preventative care, to promote well-being, and improve people lifestyles.Additional aims include providing new means of delivering health, social and informal care and to unlock new markets in social innovation, service innovation and wellness, enabled by technologies and services delivered at scale.The programme is ambitious and aims to impact the lives of nearly 170,000 people across the UK by 2015.

Figure 1. Overview of the dallas programme
The dallas programme aims to go beyond local validation of pilot studies and focus on the delivery (and evaluation) of large scale implementation of independent living technologies and services.This will include understanding the impact on individuals and on how care is delivered.
The transformation expected though the dallas programme will present both substantial opportunities and challenges for evaluation.A substantial challenge will be to identify objective measures of the impact of dallas on its target communities (individuals, families, carers, patients, care professionals etc.), on health and social care delivery systems and on the U.K. economy.To support transformation processes within health and social care delivery systems, the computing and Interaction Design community must continually strive to identify, describe, understand and reassess the factors causing users to adopt or reject technology (Bouamrane et al., 2011).From a human computer interaction (HCI) perspective, it is important to identify appropriate methods and techniques for measuring both usability and acceptability of individual technologies as well as finding innovative ways to capture user experience (UX) when interventions are dispensed through a wide service delivery and embedded in peoples lives, homes and communities rather than in the research lab.
In this paper, we briefly review the background to the programme in section 2. We review the preliminary evaluation framework in section 3 and discuss a range of outcome measures and toolkits in section 4. We conclude with a discussion and directions for future work in section 5.

Health & Social Care Policy Context
Existing models of care are unsustainable in the long term -both economically and in terms of the availability of a competent workforce to cater for an increasing elderly population with complex needs (O'Sullivan et al., 2010).As a result, policymakers are now increasingly supporting the development of nation-wide telehealth, telecare, assisted living and healthy-living technology programs (WHO, 2011).Part of the agenda in driving changes in the delivery of care is to promote coherent nationwide policies focused on prevention, wellness and community-based care.New models of care seek to reduce the reliance on acute services while supporting delivery of care and healthy living within the community, including developing telehealth and telecare programmes.Large scale deployment however remains a challenge due to the complexity of needs, issues with interoperability of systems and the fragmentation of the e-health industry.
The dallas programme is anchored within this broad policy context in the UK.In particular, it acknowledges social innovation, service redesign and innovative technologies as key to future sustainable social and care delivery models and independent living and lifestyles.One of the challenges within dallas is how to evaluate the impact of such a complex and large scale deployment of a myriad of services to a wide range of end users and consumers.

Evaluations of Large Scale Tele-Health Interventions
The traditional gold standard method for evaluating the effectiveness of complex interventions in health care has been the randomised control trial (MRC, 2000).Other methods also need to be considered due to the resources required for implementing RCTs in large scale population health and wellness interventions.
The importance of the context of service delivery and implementation processes was recognised in the revised Medical Research Council Complex Intervention Framework (Anderson, 2008) and other researchers have also highlighted the importance of analysing implementation processes in order to identify the factors which promote the success -or indeed failure -of eHealth interventions (e.g.Finch et al., 2012).
An evaluation of the Whole System Demonstrator (WSD) used a cluster randomised trial involving around 3230 people with diabetes, chronic obstructive pulmonary disease or heart failure across 179 general practices in 3 areas of England (Newham, Kent and Cornwall) (Bower et al., 2011).Results of the trial suggested that the usage of telehealth ("the remote exchange of data between patients and healthcare professionals") was associated with lower mortality and emergency admission rates (Stevenson et al., 2012).

SCOPE OF THE DALLAS EVALUATION
In addition to measuring health and wellness outcomes, the dallas evaluation will also explore how care services are designed and delivered with and for the user.The evaluation will include the assessment of care technologies and services delivered at scale as well as looking at best practices for person-centred design.It will also strive to identify processes by which new technologies and services become used, accepted and embedded into routine practice and people's lives.
The dallas programme has a strong focus on cocreating services with members of the community and delivering lifestyles that both enable and empower people rather than focusing too narrowly on health related outcomes.The framework will be robust, collaborative and pragmatic.
The evaluation of the impact of the dallas programme will include looking at: Benefits to the individual including friends, family and informal carers (ii) Benefits to the system including health and social care delivery and practices, housing, lifestyles and other cultural and organisational factors.

(iii)
Benefits to the UK economy The evaluation of (i) and (ii) (impacts on systems and individuals) will be undertaken by a multidisciplinary team in the University of Glasgow Institute of Health & Well-being and Newcastle University Institute of Health & Society.The evaluation of (iii) (impacts on the UK economy) will be undertaken by Databuild Research and Solutions Ltd 2 .
The dallas programme has identified two additional thematic streams as critical factors to effective delivery: lifestyles and interoperability (see Figure 1).Impacting on people's lifestyles will be a key factor to the success of dallas while technical interoperability will be essential to enable the nationwide scalability of the programme.How to evaluate the impact on lifestyles and the interoperability of technologies and services are therefore an integral part of the dallas programme.The i-focus community will be specifically addressing issues around interoperability of services and technologies.
We describe some of the identified challenges for the evaluation of dallas in Table 1.

HCI PERSPECTIVES & MEASURES TOOLKITS
Measuring the impact of services and technologies on people's lives is a significant challenge.HCI research has often focused on the usability of individual interactive products (such as independent living technologies) and more recently on the overall User Experience of technologies and products (Hassenzahla & Tractinsky, 2006).
While UX can address some of the evaluation aims of dallas, many aspects of the programme are concerned with how people's lifestyles are affected by services and technologies rather the technologies themselves.The evaluation team will be using a range of methods and tools for evaluating the impact of dallas on 'Lifestyles' (which includes measures of Choice, Collaboration, Control, Contribution, Connectedness, and Community).We propose creating two key toolkits in order to achieve this.
Given the novel and progressive nature of the dallas programme, it is important to share case studies, success stories and lessons learned on an regular basis throughout the programme.We propose to achieve this via user stories, video case studies and social media outlets.Knowledge transfer and sharing, as well as open innovation will be a key features of the programme.
An 'outcomes' toolkit will provide a set of validated indicators and possibly purposely-designed measures where required.It will include standard quantitative measures used in HCI, such as perceived ease of use, perceived usefulness, user satisfaction and acceptability, and health services research, such as mortality rates, hospitalisation, EQ-5D and disease specific measures.In addition, it will also include qualitative measures of wellness, well-being and Quality of Life (QOL) and perceived levels of control, connectedness, community, collaboration, choice and contribution.The user stories and outcomes toolkit will be developed in conjunction with experts and the dallas communities.

DISCUSSION
This position paper highlights some of the challenges that a large scale, complex innovative novel programme such as dallas presents for evaluation.eHealth and eCare are epistemologically complex and methodologically diverse.
Developing an evaluation of such interventions and understanding what works and why and trying to trying to establish causal relationships is complex.A simple "successionist" model in which causality can be ascribed to a simple effect is not possible and we cannot example simply link a causal outcome to a particular innovation.We have to develop what Pawson et al (2005)  Dallas is based on the hypothesis that innovative technologies can enable service innovation and result in improved wellbeing and that technologies and services can be made available to enable independent living.The success of dallas is dependent not just on the technology but also through the efforts of the stakeholders and an evaluation of their work is integral to the process.Different groups of actors, or 'relevant social groups', attach different meanings and problems to the same technology.Technical objects cannot exist without the social interactions among and indeed within, different social groups.In turn, each of the social groups and indeed each of the actors therein, attach different meanings and problems to the same technology.The implication is that if a technology is particularly flexible, it could support these different meanings simultaneously.Part of the evaluation will also involve an exploration of the different actors involved, their motivations and their actions.Hence, in part, the structure of this evaluation will be to reveal some of the different meanings attached to tele-health and tele-care.Dallas means different things to architects of the policy, to those that run the communities, to the practitioners and managers and to those that live in the communities themselves.Any evaluation must take into account the cumulative effects of all of these mechanisms as the programme unfolds.

CONCLUSION
The HCI community has much to offer in terms of defining and designing methods and measures for the evaluation of the dallas programme.Understanding technologies and how they are integrated and accepted (or not) into people's lives is central to this evaluation.Understanding how to better evaluate products and services 'in the wild' and how to capture user experience both objectively and subjectively are all key to developing a framework that can demonstrate what works and what doesn't work in the large scale roll out of independent living technologies such as in the dallas programme.
The dallas evaluation framework will continue to be refined in conjunction with the dallas communities, and service providers.As the framework evolves, the toolkits will be shared with the communities and iteratively inform the evaluation of dallas.

Table 1 :
Challenges in the Evaluation of dallas 2 http://www.data-build.co.uk/ term a generative model of causality and through this identify and contextualise underlying mechanisms.It is about identifying what changes a particular technological intervention produces for whom in what circumstances.