A Review of Apps and Websites for Promoting Mental Wellbeing: Findings and Initial Recommendations for Design

Mental wellbeing is the combination of feeling good and functioning well. Digital technology widens the opportunities for promoting mental wellbeing, particularly among those young people for whom technology is an ordinary part of life. This paper presents an initial review of publicly available apps and websites that have a primary purpose of promoting mental wellbeing. The review was in two stages: first, the interdisciplinary research team identified and reviewed 14 apps/websites, then 13 young people (7 female, 6 male) aged 12–18 years reviewed 11 of the apps/websites. Overall, the reviewers’ views were positive, although some significant criticisms were made. Based on the findings of the study, initial recommendations are offered to improve the design of apps/websites for promoting mental wellbeing among young people aged 12–18 years: highlight any age limits, provide information on mental wellbeing, improve findability, ensure accessibility on school computers, and highlight if young people were involved in design.


Terminology
A range of terms and definitions are used to describe mental health, reflecting different theoretical models. We therefore begin by briefly defining the terms used in this paper. Mental health is a broad concept that covers both mental wellbeing and mental health problems; Mental wellbeing comprises two main elements-feeling good and functioning well; Mental health problems refer to symptoms that meet the criteria for clinical problems-diagnosis of mental illness or symptoms at a sub-clinical threshold, which interfere with emotional, cognitive or social function (NHS Health Scotland, 2010). This study draws on the dual continuum model of mental health, which posits that mental wellbeing and mental health problems are related but distinct dimensions (Keyes, 2014) ( Figure 1). On these continua, mental wellbeing ranges from a low to a high level and mental health problems from absence through mild to severe clinically diagnosable illness. The dual continuum model recognises that good mental health is more than the absence of mental health problems, and illustrates that it is important to study mental wellbeing in its own right.

Mental Wellbeing
Mental wellbeing is the combination of feeling good and functioning well: feeling good incorporates the positive emotions of happiness, contentment, interest, engagement, confidence and affection; functioning well involves the development of one's potential, having some control over one's life, having a sense of purpose and experiencing positive relationships (Huppert, 2009). A range of factors influence our level of mental wellbeing including genetic, environmental and socio-cultural factors, life events and physical health. However, evidence indicates that our actions may have a greater influence (Aked et al., 2008). It is therefore important to look at possible ways for individuals to take action to promote their mental wellbeing-this paper focuses on the use of digital technology. Digital technology is an ordinary part of life for many young people. In the UK, almost all 12-24 year olds (98%) go online, and the majority of 12-15 year olds (79%) and 16-24 year olds (93%) use a smartphone (Ofcom, 2016(Ofcom, , 2017. As well as improving positive feelings towards and positive functioning in life, evidence indicates that promoting mental wellbeing could also reduce the prevalence of common mental health problems (Huppert, 2009;Keyes, 2014). In the UK, this is especially important at a time when adolescent mental health services are extremely stretched, and schools lack confidence and skills to prevent mental health problems in pupils (Ellis, 2017). Young people with poor mental health are more likely to have poor educational attainment and employment prospects, social relationship difficulties, physical health and substance misuse problems, and to become involved in offending (GOV.UK, 2013). Further, mental health problems experienced as a young person tend to persist into adulthood (Kessler et al., 2007) and in turn contribute to substantial economic burden (Mental Health Foundation, 2015).

The Study
This study continues our earlier research on digital technology for promoting mental wellbeing among young people (12-18 years), which involved focus groups with young people attending a local secondary school (Taylor et al., 2017). A key finding was that participants indicated a preference for digital sources of support for mental health, consistent with large-scale research (Young Minds, 2014), but a limited knowledge and use of what is available. This limited knowledge extended to the school staff. The current study continues this line of investigation with a short review of digital technologies that have a primary purpose of promoting mental wellbeing. The review was in two stages: first, the interdisciplinary research team identified and reviewed 14 apps and websites; then 13 young people aged 12-18 years reviewed 11 of them. This paper contributes to the body of research within HCI on design for mental wellbeing. The main contributions are:  Findings from the review, including the views of a small sample of young people on publicly available digital technologies for promoting mental wellbeing, which is an area that is particularly under-researched.  Based on the findings of the review, initial recommendations to improve the design of apps/websites for promoting mental wellbeing in young people aged 12-18 years.
An account of the research team's review is included in a previous paper by the authors (Taylor et al., 2017). It is also reported here as the findings are an integral part of the recommendations for design.

Mental Health and Young People
HCI research on mental health and young people has mostly focused on design for mental health problems (e.g. Coyle et al., 2011;Lederman et al., 2014;Wadley et al., 2013). There have been fewer studies with an emphasis on promoting mental wellbeing. Thieme et al. suggest this is partly due to the concept of, and strategies to design for, mental wellbeing being less well understood both in healthcare and HCI; but note that the need to gain a better understanding of mental wellbeing and how to design for it has become an increasing concern in HCI research . In the 'Spheres of Wellbeing' project, Thieme et al. take a holistic approach to mental health. The project involved the design of technology (Spheres) to support the treatment and promote the mental wellbeing of women (18+ years) with a complex mental health condition (Thieme et al., 2016). The Spheres were evaluated as having a positive impact on the women's mental wellbeing and were considered a useful addition to existing services by hospital staff (Thieme, 2015). Positive computing, the design of technologies that promote psychological wellbeing, is also a growing area of research in HCI (Calvo and Peters, 2013). In the first study to take a participatory design approach to developing positive computing technologies for young people, 434 ideas were created by participants (11-12 years), eight of which were prototyped (Yarosh and Schueller, 2017). The ideas revealed participants' understanding of positive psychology concepts, e.g. mindfulness, and their views on how best to promote the practice of these concepts-notably, over one third of the ideas were apps. Other recent studies have investigated mental wellbeing among college students, and highlighted considerations and implications for the design and research of technologies for self-tracking (Kelley et al., 2017) and for leveraging social media data to improve the collective mental wellbeing of college campuses (Bagroy et al., 2017).

Reviews of Apps for Mental Health
There have been numerous reviews of apps for mental health, many of which have reported the potential for digital technology to play a valuable role in the field of mental health, while highlighting limited evidence of effectiveness. For example, a systematic review of the research evidence supporting the efficacy of mental health apps concluded they have the potential to be effective and may significantly improve access to treatment, but there are few evidence-based apps available in the public domain (Donker et al., 2013). That review included articles published between 2008 (when the first app was launched) and 2013. The current situation seems to be much the same and it is largely unclear what constitutes a minimum acceptable standard of evidence (Leigh, 2016). NHS England is attempting to address this issue by working with partners to develop a process for assessing apps, and standards and guidance information for developers. Apps that are considered good quality in terms of safety, experience and effectiveness will be added to the recently launched NHS Digital Apps Library (Beta) website for patients and healthcare professionals to access.
There have been fewer reviews of apps/websites for mental wellbeing specifically. In particular, there has been a focus on reviews of apps for mindfulness. Mindfulness is the practice of being attentive to and aware of what is taking place in the present, which can promote wellbeing. A systematic review of publicly available mindfulness apps reported that the apps reviewed (n=23) had an overall acceptable level of quality (Mani et al., 2015). An efficacy study was identified on just one of these apps, Headspace, which showed significant increases in elements of wellbeing. Headspace also had the highest overall rating for quality followed by the Smiling Mind app; both of these apps are included in our review.

Recommendations for Design
HCI research has contributed recommendations and guidelines for the design of digital technologies for mental health. Doherty et al. presented an initial set of guidelines for the design of mental health technologies, extracted from the literature and from the authors' research experience (Doherty et al., 2010). However, as the authors point out, in terms of the scope of the guidelines there is a focus on interventions involving a therapist. The authors also discuss the difficulty, if not impossibility, of producing a canonical set of guidelines that applies to all applications in the area. Sixteen evidence-based recommendations for developers to design better apps were formulated based on a literature review of mental health apps (Bakker et al., 2016). This study supports several of these recommendations: providing activities, presenting mental health information and conducting experimental trials to establish efficacy. However, Bakker et al. (2016) had a slant towards preventing mental health problems rather than promoting mental wellbeing. A study that explored the views of young people on the use of mental health apps, and their feedback on a prototype app developed by the authors, concluded that this type of tool is acceptable to adolescents, but identified safety, functionality and engagement as key concerns (Kenny et al., 2016). The results of this study support the latter two themes: collectively, the reviewers considered it was important for apps to present mental health information and to offer something unique. However, the study by Kenny et al. (2016) was limited to the views of young people aged 15-16 years.

METHOD
A review was conducted of digital technologies that have a primary purpose of promoting mental wellbeing. The review was in two stages: first, the interdisciplinary research team identified and reviewed a set of apps/websites; then a group of young people reviewed a subset of the apps/websites.

Stage 1: Research Team Review
Due to time constraints, a limited number of sources were searched for publicly available digital technologies that have a primary purpose of promoting mental wellbeing. The main source was Aye Mind. Aye Mind is a EU-funded project led by a NHS Board in Scotland as part of its strategy to promote child and youth mental health. The project website includes a directory of digital technologies identified, though not endorsed, by the Aye Mind team to support young people's mental health. The second source was the NHS Choices website, which included a more limited directory of online mental health services approved for use by NHS England. Technologies that our earlier focus group research participants reported that they and some of their peers used were also included.
Sixty-seven digital technologies were collected. Of these, 14 technologies met the inclusion/exclusion criteria for the review-all of which were either an app, with or without a web version, or a website ( Table 1). The inclusion criterion was technologies with a primary purpose of promoting mental wellbeing-as opposed to preventing or treating mental health problems. The exclusion criteria were technologies targeted at people aged less than 12 years or more than 18 years. The 14 apps/websites were reviewed in detail by the four members of the research team from their specialist perspectives: Interaction Design, Public Health, Health Psychology and Applied Psychology-involving each member individually using (trialing) the app/website thoroughly and making notes, followed by a group discussion. Finally, the Child and Adolescent Mental Health Service (CAMHS) in our area reviewed and validated the findings. CAMHS are specialist NHS services that help children and young people with their mental health.

Stage 2: Young People's Reviews
Participants in our earlier focus group research who were still enrolled at the school (n=27) and were interested in ways to promote mental wellbeing, were invited to give their views on 11 of the apps/websites. Each participant individually used (trialled) an app/website thoroughly and wrote their views on a review form. On the advice of the CAMHS that collaborated on the review, three social networking apps (Moodbug, Silent Secret, TalkLife) were omitted due to risk factors such as online peer victimisation. As the study was conducted a year after our focus group research, the then S1 and S6 year groups had moved up to S2 and left school respectively. Therefore a few young people from the current S1 year group were also invited to participate. Each young person was given an information sheet that explained the purpose of the study and the nature of their involvement, a summary of the 11 apps/websites, a review form, and a consent/assent form for their parent/carer to read, and if agreeable, to sign and return to the school. For those young people whose parents consented to their participation, a meeting was held to: allot the apps/websites, explain the review form, answer any questions, and obtain assent from each young person. Participants mostly chose which app/website they reviewed. Some participants were asked to review a particular app/website, or more than one app/website, to ensure that the whole set was reviewed. Participants completed their review in their own time within a twoweek timeframe, using their own computer or smartphone or a school computer, and returned it to the research team. Due to scheduling difficulties, it was not possible to conduct a group discussion. As a token of appreciation, each participant received a certificate of thanks and a £10 Amazon gift card.

Review Form
A 4pp A4 paper form (A3 size paper folded in half) was provided for participants to write their review on. At the start of the form, six questions on aspects of engagement, usability, aesthetics, perceived impact and satisfaction were printed as prompts to get participants thinking: The written reviews were sorted in a spreadsheet file under the broad headings of engagement, usability, aesthetics, perceived impact and satisfaction. A small group of young people (separate to the study participants) previewed the review form, all of who considered it easy to understand.

Stage 1: Research Team Review
Two of the apps (Ginsberg, Headspace) have no age limits. The remaining apps/websites are targeted at young people. Overall, the research team considered the apps/websites to be helpful and, for the most part usable and visually appealing with appropriate language. Some of the good information on a few websites (Childline, The Mix and Young Scot) was harder to find. However, limitations were identified: a lack of practical tools within the information-based websites, a lack of practical tools within several of the apps, a lack of information on mental health within all of the apps, and limited evaluation for effectiveness. The provision of practical tools is important to support young people to translate what they are reading into exercises that they can experience and use in times of need. To our knowledge, only two of the apps had thus far been evaluated for effectiveness: Headspace (Howells et al., 2016) and In Hand (Simons et al., 2015), which is consistent with reports of a high availability of apps but low evidence base of effectiveness (Leigh and Flatt, 2015).
For example, Ginsberg is an activity and mood diary that helps individuals identify sources of stress and anxiety, and monitor the impact of changes made to their behaviour and lifestyle. However, it does not provide immediately accessible suggestions of strategies that could be employed to address the identified issue. It is therefore essential that the user has prior knowledge-or can find out from an alternative source-what type of changes would be effective and how to make them. Similarly, SafeSpot promotes the development of positive strategies for coping with adverse life events by encouraging young people to create a personalised list of coping skills. However, it does not provide any practical information within the app to generate the list and so relies on the young person having external knowledge of different coping skills and how to put them into practice. SafeSpot also does not provide any information on mental health within the app.
Another example is In Hand, which suggests activities to help young people in a moment of stress or low mood. The app is intended as a simple tool with a specific, focused purpose. However, it neither provides practical tools to help the young person better understand their feelings nor information on mental health. By contrast, YoungMinds and ReachOut.com provide a great deal of useful information on mental health including medication and real stories (YoungMinds) and information on a broader range of topics such as physical health, alcohol and other drugs (ReachOut.com). However, neither website provides immediately accessible practical tools to support young people to apply the information they have read.
Despite the increasing recognition of schools as important settings for promoting mental health in young people, for the most part, the apps and websites sampled in the review do not explicitly link with school learning in mental health. Only two of the apps have an associated school programme: SafeSpot and Smiling Mind. Alongside the SafeSpot app, the SafeSpotter Programme trains selected older students to be first points of contact in schools for younger people experiencing mental health problems, as an alternative to approaching an adult. Alongside the Smiling Mind app, the Smiling Mind Education Programme is a series of age specific lesson plans mapped to the Australian Curriculum that can be used to support existing wellbeing, pastoral care and training. Two of the information-based websites included in the review, ReachOut.com and YoungMinds, offer courses and resources for education professionals.

Stage 2: Young People's Reviews
Thirteen young people completed a review (7 female, 6 male), with at least one young person from each school year group participating except for S4 (Table 2). One participant (R6) had prior experience of one of the apps (Headspace), which they reviewed. Another participant (R1) could not complete their review because they were unable to sign up to their app of choice (Ginsberg). Indeed, it appeared that without notice or explanation Ginsberg by the Scottish Government was no longer supported. The remaining apps/websites were reviewed by at least one participant: SafeSpot, Childline, ReachOut.com, The Mix, YoungMinds, Young Scot and Smiling Mind (one review each); In Hand (three reviews); Headspace (two reviews); and Stressheads (two reviews).

Engagement
This website is probably aimed at older kids because children my age probably wouldn't start thinking about sex, relationships, smoking or taking drugs. (R7: ReachOut.com) Overall, participants' comments with respect to aesthetics were very positive. In particular, participants liked the appearance of several of the apps/websites, describing the colours as 'bright ', 'colourful' and 'appealing to the eye' (In Hand, SafeSpot, Smiling Mind, The Mix, Young Minds); and the layout as 'clear ', 'very nice' and 'attractive' (In Hand, ReachOut.com, The Mix [Stressheads] good and a little confusing'. R4 expressed an overall dissatisfaction with In Hand, commenting 'I felt multiple times when I was using the app that I was missing something or that there must be more.' R4 also felt that young people would not 'feel safe' using In Hand to write down their feelings, as the app cannot be password protected. Indeed, a guideline from Doherty et al. (2010) is that data must be perceived to be secure.

RECOMMENDATIONS
Overall, the research team and young participants were positive about the digital technologies reviewed. Based on the findings of the review, initial recommendations were formulated to improve the design of apps/websites for promoting mental wellbeing in young people aged 12-18 years. This set of recommendations extends existing guidance outlined in the Related Work section, providing a useful addition for HCI designers and researchers working in the area of mental health and young people to consider.

Highlight any Age Limits
During the process of conducting the study, the research team identified an issue related to the target age of the individual apps/websites. Firstly, it was often not apparent whether age limits applied, and if so what they were, requiring contacting the individual organisations; yet this is important information to ensure that young people are accessing age-appropriate content. Secondly, the age limits (where applicable) varied across the individual apps/websites, reflecting the varying definitions of 'young people' in relation to age. Thirdly, the age limits were often quite broad. For example, Young Scot is targeted at 11-26 year olds. One of the young participants, R7, commented that the website they reviewed, ReachOut.com, was probably not intended for young people their age (12-13 years

Provide Practical Tools and Information on Mental Wellbeing
A criticism by the research team and several of the young participants about the apps/websites reviewed related to a lack of information on mental health within all of the apps. Providing such information is important: it can provide justifications for performing recommended actions or activities, which in turn could lead to more young people taking steps to promote their mental wellbeing; and it can improve understanding about mental health, which in turn is associated with a reduction in stigmatising beliefs about those with mental health problems and an increase in help-seeking (Bakker et al., 2016). A simple solution to the lack of information on mental health within the apps included in the review is to provide access to an information website. For example, the Stressheads app links to articles on its developer's website (The Mix) as well as other websites for information on mental health. An additional criticism by the research team related to a lack of practical tools within the information-based websites and several of the apps. Practical tools are important to support young people to translate what they are reading into exercises/mechanisms that they can experience and use in times of need. What these practical tools might look like is dependent on the purpose and target outcomes of the particular app or website. However, to illustrate the point, the In Hand app included in the review is designed to help in a moment of stress or low mood. It incorporates a basic tool for listening to music, which should be helpful, given that music listening has been directly linked to mood improvement (Bakker et al., 2016). However, the app design could be improved by incorporating a practical tool to help young people understand the factors that affect their mood and stress levels.

Improve Findability
A key finding of our earlier focus group research is that the young people who participated are positive about apps/websites as a source of support for mental health (which is consistent with other research e.g. Kenny et al., 2016 andYoung Minds, 2014), but have a limited knowledge of what is available. Indeed, the Children's Commissioner for England has noted while there are some good websites for mental health, it is a 'matter of luck' whether children find them (Richardson, 2015). Similarly, research by Neal et al., (2011) found that good quality mental health resources are not reaching young people aged 18-25 years at acceptably high levels.
A key finding of this study is that the young participants are also positive overall about the apps/websites sampled. An implication is that young people (12-18 years) may be willing to use apps/websites to promote their mental wellbeing, but need to be better supported to find them. Health promotion recognises that initiatives are unlikely to be successful unless people's own understandings are taken into account (Secker et al., 1999). However, there is currently limited research on: (i) how young people (12-18 years) understand the concept of mental wellbeing and the language that they use-even though it is acknowledged that the terminology of mental health is adult-centric and that young people may use a different form of language; and (ii) the terms and phrases that young people use when searching online for apps/websites for promoting mental wellbeing. A better understanding of these factors could potentially be used to improve the design of an app/website, e.g. by composing more relevant copy and metadata, helping to eliminate the element of luck in finding apps/websites that are likely to be useful. This recommendation is consistent with prior research that proposes researchers should engage with the mental models of the particular child audience before deploying a positive technology intervention (Yarosh and Schueller, 2017).

Ensure accessibility on school computers
Each website and web-based app was tested on a school computer to check that it could be accessed in advance of the young people completing their reviews. A single website, The Mix, was found to be partially blocked by the school's web filtering system and therefore required to be accessed on the participant's computer or smartphone. The Mix is targeted at young people aged 13-25 years. In particular, the section 'Get Support' and several sub-sections of the section 'Sex & Relationships' (e.g. 'Virginity' and 'Abstinence') were blocked. The 'Get Support' section includes a local services directory and one-to-one support from a trained team by phone, web chat or email. Access to this type of information is important. For example, young people are more inclined to seek help for mental health problems if they are knowledgeable about sources of help (Rickwood et al., 2007). In addition, in Scotland, where our research was conducted, 'Relationships, sexual health and parenthood' is a topic that is studied by young people as part of the national curriculum for schools. Thus, the 'Sex & Relationships' section could be a useful resource for school learning in health and wellbeing. Ensuring that every element of an app/website design is accessible on school computers, and arranging for content to be unblocked if necessary, is important to optimise its potential for use by young people, as well as adults who support the learning of young people such as teachers and school nurses.

Highlight if Young People Were Involved in Design
Overall, the views of the young participants on the apps/websites that they reviewed were clear and insightful, indicating that young people may have a valuable role in the design of technology targeted at their age group. Indeed, there is increasing literature to support a participatory approach for the design of technologies for young people's mental health (e.g. Simons et al., 2015;Wadley et al., 2013). Further, it is the democratic right of young people for their voice to be heard (Hart, 1992). During the briefing for this study and during our earlier focus group research, several participants asked whether young people had been involved in the design process of the apps/websites discussed/reviewed. However, this information was often not apparent to the research team-again requiring contacting the individual organisations and/or searching for that information online. The young participants' comments suggest that if young people are involved in the design of an app/website, displaying a prominent notification to that effect within its design may encourage more young people to explore it. For example, R4 was put-off by Ginsberg: The fact that it's Scottish Government just makes me think 'Government', adults assuming they know what's right for kids, we're the only people who know what it's like. It's like when you hear about people who left school years ago deciding, well you're not in the thick of it.

LIMITATIONS
A limitation of the review is that it did not involve exhaustive, comprehensive searching for digital technologies. Consequently, a limited sample of apps/websites was reviewed by the research team and young participants. Further, the original set of 14 apps/websites was reduced to 10 for the young people's reviews: it was not possible to trial the Ginsberg app; and on the advice of the collaborating CAMHS, the social networking apps Moodbug, Silent Secret (later rebranded to Cypher) and TalkLife were omitted. It is important to note that some young people in our earlier focus group research reported that they and some of their peers used Silent Secret and TalkLife; that both of these social networks have safeguards to protect users; and other research exploring young people's views on mental health apps has identified social interaction as an important factor (Kenny et al., 2016;Yarosh and Schueller, 2017). This points to a potential tension between what young people may want to use and what specialist services may be comfortable recommending.
An additional limitation of the study is the small sample size of young people and the small number of reviews. The number of young people who participated was lower than expected. In part, this may have been a consequence of having repeatedly approached the same young people to participate in research: the S2-S6 year groups had taken part in our earlier focus group research and all year groups had taken part in a series of related activities (not reported in this paper). Therefore, the results are considered preliminary-future studies should engage with a larger sample of young people. However, it is important to stress that the recommendations are derived from the findings of the interdisciplinary research team, validated by the local CAMHS, the young people, and the process of conducting both reviews. The recommendations are also to a limited extent grounded in prior research.

CONCLUSION
The mental health of young people is a global public health challenge. Good mental health requires the complementary activities of promoting mental wellbeing and preventing and treating mental health problems. The role of digital technology in promoting mental wellbeing is receiving increasing attention in the field of HCI. This paper presented the findings of a review of apps/websites that have a primary purpose of promoting mental wellbeing by an interdisciplinary research team, in collaboration with a local CAMHS, and 13 young people (12-18 years). Based on the findings, initial recommendations to improve the design of apps/websites for promoting mental wellbeing among young people are offered: highlight any age limits, provide information on mental wellbeing, improve findability, ensure accessibility on school computers, and highlight if young people were involved in design.