Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Despite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models.

Objective

This study aimed to develop and pilot test the Technology Evaluation and Assessment Criteria for Health apps (TEACH-apps).

Methods

Tailoring a well-known implementation framework, Replicating Effective Programs, we present a new process to approach the challenges faced in implementing app evaluation tools today. As a culmination of our experience implementing this process and feedback from stakeholders, we present the four-part process to aid the implementation of mobile health technology. This paper outlines the theory, evidence, and initial versions of the process.

Results

The TEACH-apps process is designed to be broadly usable and widely applicable across all fields of health. The process comprises four parts: (1) preconditions (eg, gathering apps and considering local needs), (2) preimplementation (eg, customizing criteria and offering digital skills training), (3) implementation (eg, evaluating apps and creating educational handouts), and (4) maintenance and evolution (eg, repeating the process every 90 days and updating content). TEACH-apps has been tested internally at our hospital, and there is growing interest in partnering health care facilities to test the system at their sites.

Conclusions

This implementation framework introduces a process that equips stakeholders, clinicians, and users with the foundational tools to make informed decisions around app use and increase app evaluation engagement. The application of this process may lead to the selection of more culturally appropriate and clinically relevant tools in health care.

Related collections

Most cited references 13

Record: found
Abstract: found
Article: found

Is Open Access

Implementing evidence-based interventions in health care: application of the replicating effective programs framework

Amy Kilbourne, Mary Neumann, Harold Pincus … (2007)

Background We describe the use of a conceptual framework and implementation protocol to prepare effective health services interventions for implementation in community-based (i.e., non-academic-affiliated) settings. Methods The framework is based on the experiences of the U.S. Centers for Disease Control and Prevention (CDC) Replicating Effective Programs (REP) project, which has been at the forefront of developing systematic and effective strategies to prepare HIV interventions for dissemination. This article describes the REP framework, and how it can be applied to implement clinical and health services interventions in community-based organizations. Results REP consists of four phases: pre-conditions (e.g., identifying need, target population, and suitable intervention), pre-implementation (e.g., intervention packaging and community input), implementation (e.g., package dissemination, training, technical assistance, and evaluation), and maintenance and evolution (e.g., preparing the intervention for sustainability). Key components of REP, including intervention packaging, training, technical assistance, and fidelity assessment are crucial to the implementation of effective interventions in health care. Conclusion REP is a well-suited framework for implementing health care interventions, as it specifies steps needed to maximize fidelity while allowing opportunities for flexibility (i.e., local customizing) to maximize transferability. Strategies that foster the sustainability of REP as a tool to implement effective health care interventions need to be developed and tested.

0 comments Cited 202 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: not found
Article: not found

A Hierarchical Framework for Evaluation and Informed Decision Making Regarding Smartphone Apps for Clinical Care

Jung Kim, Steven Chan, Shih Yee-Marie Tan Gipson … (2018)

0 comments Cited 86 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Quantifying App Store Dynamics: Longitudinal Tracking of Mental Health Apps

Mark Erik Larsen, Jennifer M. Nicholas, Helen Christensen (2016)

Background For many mental health conditions, mobile health apps offer the ability to deliver information, support, and intervention outside the clinical setting. However, there are difficulties with the use of a commercial app store to distribute health care resources, including turnover of apps, irrelevance of apps, and discordance with evidence-based practice. Objective The primary aim of this study was to quantify the longevity and rate of turnover of mental health apps within the official Android and iOS app stores. The secondary aim was to quantify the proportion of apps that were clinically relevant and assess whether the longevity of these apps differed from clinically nonrelevant apps. The tertiary aim was to establish the proportion of clinically relevant apps that included claims of clinical effectiveness. We performed additional subgroup analyses using additional data from the app stores, including search result ranking, user ratings, and number of downloads. Methods We searched iTunes (iOS) and the Google Play (Android) app stores each day over a 9-month period for apps related to depression, bipolar disorder, and suicide. We performed additional app-specific searches if an app no longer appeared within the main search Results On the Android platform, 50% of the search results changed after 130 days (depression), 195 days (bipolar disorder), and 115 days (suicide). Search results were more stable on the iOS platform, with 50% of the search results remaining at the end of the study period. Approximately 75% of Android and 90% of iOS apps were still available to download at the end of the study. We identified only 35.3% (347/982) of apps as being clinically relevant for depression, of which 9 (2.6%) claimed clinical effectiveness. Only 3 included a full citation to a published study. Conclusions The mental health app environment is volatile, with a clinically relevant app for depression becoming unavailable to download every 2.9 days. This poses challenges for consumers and clinicians seeking relevant and long-term apps, as well as for researchers seeking to evaluate the evidence base for publicly available apps.

0 comments Cited 81 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

John Torous:

ORCID: https://orcid.org/0000-0002-5362-7937

Division of Digital PsychiatryBeth Israel Deaconess Medical CenterHarvard Medical School330 Brookline AvenueBoston, MA, 02215United States1 7143359858jtorous@bidmc.harvard.edu

Journal

Journal ID (nlm-ta): J Med Internet Res

Journal ID (iso-abbrev): J. Med. Internet Res

Journal ID (publisher-id): JMIR

Title: Journal of Medical Internet Research

Publisher: JMIR Publications (Toronto, Canada )

ISSN (Print): 1439-4456

ISSN (Electronic): 1438-8871

Publication date Collection: August 2020

Publication date (Electronic): 27 August 2020

Volume: 22

Issue: 8

Electronic Location Identifier: e18346

Affiliations

[1 ] Division of Digital Psychiatry Beth Israel Deaconess Medical Center Harvard Medical School Boston, MA United States

[2 ] Bicycle Health Boston, MA United States

Author notes

Corresponding Author: John Torous jtorous@ 123456bidmc.harvard.edu

Author information

Erica Camacho https://orcid.org/0000-0002-5466-8853

Liza Hoffman https://orcid.org/0000-0002-7040-839X

Sarah Lagan https://orcid.org/0000-0001-7540-5238

Elena Rodriguez-Villa https://orcid.org/0000-0001-9154-2339

Natali Rauseo-Ricupero https://orcid.org/0000-0002-3492-5541

Hannah Wisniewski https://orcid.org/0000-0002-9931-2447

Philip Henson https://orcid.org/0000-0001-5206-9374

John Torous https://orcid.org/0000-0002-5362-7937

Article

Publisher ID: v22i8e18346

DOI: 10.2196/18346

PMC ID: 7484774

PubMed ID: 32535548

SO-VID: 449d3ae5-4128-40f5-a941-29f7c453b64c

Copyright © ©Erica Camacho, Liza Hoffman, Sarah Lagan, Elena Rodriguez-Villa, Natali Rauseo-Ricupero, Hannah Wisniewski, Philip Henson, John Torous. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 27.08.2020.

License:

This is an open-access article distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

History

Date received : 20 February 2020

Date revision requested : 8 April 2020

Date revision received : 17 May 2020

Date accepted : 13 June 2020

Comments

Comment on this article

scite_

Cited by 6

See all cited by

Most referenced authors 188

See all reference authors

Submit your digital health research with an established publisher
- celebrating 25 years of open access