Development of an iPad version of the Kessler 10+ for use in youth mental health outreach services
Gareth Furber, PhD1, Ann E Crago2, Tom D Sheppard3, Clive Skene4
1(Clinical Psychology) – University of South Australia, Adelaide, SA, 5000; 2Bachelor of Nursing – Youthlink, Women’s and Children’s Health Network, SA Health, GP Plus Health Care Centre Marion, 10 Milham Street, Oaklands Park, Adelaide, SA, 5046; 3Registered Nurse (Mental Health) – Youthlink, Women’s and Children’s Health Network, SA Health, GP Plus Health Care Centre Marion, 10 Milham Street, Oaklands Park, Adelaide, SA, 5046; 4Master of Psychology – CAMHS Executive, Level 1, 55 King William Road, North Adelaide, SA, 5006
Corresponding Author: E gareth.furber@unisa.edu.au
Journal MTM 4:1:20–24, 2015
In this case report we describe the development and early trialling of an iPad application replicating the Kessler 10+ (K10 +), a widely used brief measure of psychological distress. The application was the result of a collaboration between a youth mental health outreach service (Youthlink), a private application developer (Enabled) and local health service IT support. Therapists reported greater engagement with the iPad version of the K10+ compared to the pen/paper version and described how the application assisted them to collaboratively reflect with consumers on treatment progress.
Case study
Youthlink is an Australian Child and Adolescent Mental Health early intervention service (CAMHS) for 16 to 19-year olds who are experiencing mental health difficulties or having trouble making the transition between CAMHS and adult mental health services. Youthlink services approximately 150 young people a year through the work of 6-7 staff (nursing, psychiatry, social work). The clinical presentation of these young people is complex. They present with Axis 1 disorders such as depression, psychosis and anxiety, have diagnostic comorbidity, and report histories of abuse, trauma, family or peer group problems, school/work issues, harmful substance use and homelessness.
Youthlink consumers can be difficult to engage through traditional channels (e.g. clinic based consultations, home phones, and reminder letters). Thus much of the work consists of connecting with these consumers in flexible ways, on the run, via home visits, last minute appointments, and mobile phone calls. This flexibility ensures that these clients are regularly monitored as they exhibit a number of high-risk behaviours (e.g. impulsivity, self-harm and substance use) as well as high risk exposure (e.g. homelessness, legal issues, financial problems).
One of the instruments that Youthlink use to monitor the outcomes of consumers is the K10+. The original K101 is a 10-item consumer-reported questionnaire intended to yield a global measure of distress based on questions about anxiety and depressive symptoms that a person has experienced in the most recent 4 week period. The “+” refers to a version with 4 additional questions relating to degree of work impairment and health service utilisation. The K10 was developed by Kessler and colleagues for use in the US National Health Interview Survey as a screening scale for non-specific distress. Validation studies have supported its appropriateness as a brief measure of psychological distress in Australian samples2,3.
Until recently, Youthlink administered the K10+ as a pen/paper instrument. In outreach work this process had a number of limitations. The instrument was unengaging to consumers and therapists had to remember to carry paper copies and conduct hand scoring. More importantly, therapists had to remember to carry a consumer’s previous K10+ s if wanting to discuss progress over time. When the first version of the iPad™ was released in 2010, the team proposed to develop a tablet-based version of the K10+ to address these limitations. A tablet version of the K10+ would be engaging, readily available, automatically scored, and much better suited to collecting multiple K10+ over the course of treatment. This case report outlines Youthlink’s development of the K10+ iPad application and the early experiences of the therapists in using the application in clinical practice.
Development of the application
Development and trialling was conducted in 5 stages. Stage 1 (Consultation) involved working with local Information Technology personnel to develop a functional specification document outlining the expected screens, functionality, security features and basic design of the application. Stage 2 (Approvals) involved confirmations from local and state clinical reference groups that the project adequately protected consumer privacy, and from local and state health IT that the application met guidelines for use within existing health IT infrastructure. Stage 3 (Tendering) involved tendering the project out to private developers and receiving approvals to release funds for iPadsTM and software development. Stage 4 (Development) was led by a local application developer called Enabled (www.enabled.com.au). Enabled used the functional specification document from Stage 1, which was further supplemented through direct consultation with Youthlink and contributions of their own ideas. Enabled used an agile development process whereby iterative design and app decisions were showcased regularly to the team before making adjustments/changes and moving to the next step of development. Early versions of the application were distributed to team members to trial in the field, to identify useability issues and software bugs. Finally in Stage 5 (Field trial), all six Youthlink therapists received iPads and a working version of the application to use in practice. The field trial went for 6-months, 3-months to allow familiarisation with the application, and 3-months using as standard practice. Therapists’ experiences with the application were gathered during a focus group, 6 months after the cessation of the field trial. In addition, K10+ collection rates during the second 3-month period of the field trial were compared against the same period in the previous year (when iPads were not being used).
The application
Screenshots from the application are presented in Figure 1. The application is password protected (A). After sign-in, the therapist searches for the relevant client and can either review previous K10+ or launch a new one. Upon launching a new K10+, the tablet is handed to the client. Each item of the K10+ is replicated on the device, with one question per page and ability to move backwards and forward through the questionnaire. As per the standard K10, clients are asked to answer in relation to the last 4 weeks. The 10 Likert scale distress items (rated “none of the time” through to “all of the time”) are represented with large blue button choices (B). Items relating to days unable to work/restricted in working and visits to a doctor/health professional were visualised using a grid of 28 dots (4 lines of 7), reflecting the last 4 weeks (C). Each dot selected represented a day. This visual metaphor provided a neater way for clients to think back over the past 4 weeks and report their work capacity and medical appointments. This visual metaphor reflected one of a number of suggestions the team at Enabled made in the development of the application.
Figure 1: Screenshots from the iPad application
Upon completion, the K10+ is automatically scored and represented as a single score in a progress chart (D). If multiple K10+ have been completed by the client, all are displayed along a timeline allowing for a visual representation of a consumer’s distress scores over time. Any individual measurement instance can be “exploded” to examine the client’s scores across the 14 items (E). The 10 distress items are organised into four categories (negative affect, nervous, agitation fatigue) based on a factor analysis by Brooks4. The use of these categories helps consumers understand the different symptom clusters that make up their distress. In addition to the current score, this exploded chart shows how that item has changed since the last completion of the K10+. This allows a more fine-grained analysis of which areas are showing improvements or worsening. Finally for data entry purposes, the app can generate a historical table of a client’s answers over time (F), and data for all clients on the iPad can be exported through iTunes to a csv file for further analysis. The app does not require a network connection to function.
Therapists’ positive experiences of the application
Therapists reported that the application was well received by consumers who found it easy and quick to use, well-designed and liked the visual representation of their therapy progress. Therapists found themselves using the application collaboratively, with consumers and therapists reviewing the results of assessments together and incorporating the results into the treatment. Therapists noted a number of benefits of this process. It increased consumers’ emotional self-awareness and vocabulary, giving them words for their different symptoms. It provided a more concrete basis for treatment decisions, identifying which symptoms were more troublesome. It provided useful evidence for consumers who felt they were not making progress but whose symptoms were improving. It acted as a clearer prompt for therapists to act on declining scores, inquire about specific symptoms or investigate inconsistencies in clients’ report (e.g. showing high distress in person but low distress on K10 +). Finally it provided a mechanism for consumers who were uncomfortable talking about their feelings to express how they were doing in a less confronting way.
Therapists reported being much more engaged with the application over the pen/paper measure which reflected in their use of the application. Therapists were more likely to use the K10+ at intake to get a baseline level of functioning. For example, 57% of admissions to the service during the 3-month audit when using the iPad version had a K10+, compared to 27% of admissions in the previous 6-months. They were also more likely to use the application at review points during therapy where they would discuss with consumers their progress over time. The convenience of having the application on the iPad (which they carried with them like a diary) was a primary driver of these benefits. In addition, other features of the application such as its design (consumers “thought it was cool”), and use of progress charts to show change in K10+ scores over time, made the application highly suited to repeated measurement and considerably more attractive to use than the pen/paper version.
Therapists’ negative experiences of the application
The application had a number of functional limitations reflecting the budget of the project and the organisational challenges of embedding new IT technologies within existing systems. These included not integrating with therapists’ calendars and the client record system, features identified by therapists as crucial for the long-term adoption of tablet-based outcome assessments. Additionally, for security purposes the application required secure login, de-identified codes for individual clients, and a search-based method for accessing client information from the device. These organisational requirements made some aspects of using the application, most notably, finding and sorting through clients, somewhat cumbersome.
A significant frustration with the application was due to provisioning issues. Provisioning refers to the means by which therapists download the application to their device. As the application was not intended to be released in the Apple store, provisioning of the application was initially handled by Enabled and then by the IT department of the local health service. Having not been involved with provisioning prior to this project, the IT department initially struggled with providing a reliable means of accessing the application, which meant all therapists experienced the application ceasing to work during the 6-month trial leading to lost data. At the time of writing, these provisioning issues had been solved.
Finally, using the application required therapists to adopt the iPad as part of their daily workflow (e.g. email, calendaring). Three of the six therapists found this easy and embedded the iPad and application in their standard practice and continued to use it post-trial period. However three found it was a poor fit to their workflow and used the iPad (and hence application) only intermittently after the field trial.
Discussion
The Youthlink team set out to develop an iPad version of the K10+ to overcome the barriers of pen/paper measurement in youth outreach work. The development process engaged clinicians to rethink their use of a standard routine outcome measure (K10 +) and imagine ways of making assessment a more collaborative process. The resulting application was engaging, clinically useful, convenient, and a valid replacement for the pen/paper K10+. Feedback from therapists was that the use of the application changed their behaviour regarding use of the K10+ thus creating a more collaborative measurement process which had tangible benefits for clients in terms of mental health literacy, communication of distress, therapy planning and monitoring. In this respect, our experiences are illustrative of the purported benefits of routine outcome assessment5 and consistent with positive reports of tablet-based assessment in other fields6,7.
Two aspects of translating the K10+ to the iPad stood out. The first was the ability to harness the touchscreen interface to increase engagement. Features such as single question per page, larger text, clearer response options, dynamic colour contrasts, assistive visual cues for difficult questions, and responsive touch made completing the K10+ on the iPad faster, more intuitive and more enjoyable. The second aspect was the application’s ability to automatically generate and display feedback charts that provided therapists and consumers with information that could be used to structure their treatment. This process was facilitated by the wide viewing angles on the iPad screen which allowed therapist and consumer to review charts simultaneously. A number of authors have identified feedback as a crucial ingredient in improving compliance in outcome measurement8–10.
Adoption of new technologies into mental health services is aided by case studies of services that have experimented with implementing new practices. For example in previous work we explored the use of Short Messages Service to foster better communication with consumers11. This work was promoted nationally through the professionals sections of the Reachout website by inclusion as a case study for training for mental health professionals12. We hope that this brief case study similarly inspires mental health services to think about how mobile technologies can be used in clinical practice. Future work should explore whether introduction of such applications reliably increases therapists’ use of routine outcome measurement.
Acknowledgements
The authors would like to acknowledge the Mental Health Unit within the Department of Health in South Australia for the financial support of this project. We’d also like to thank the team at Enabled for their professionalism in building the application and Greg Chambers and Jon Holloway for local IT support and helping get the project off the ground. All those who contributed significantly to this paper are reflected in the authorship list.
References
1. Kessler RC, Barker PR, Colpe LJ, Epstein JF, Gfroerer JC, Hiripi E, et al. Screening for serious mental illness in the general population. Arch Gen Psychiatry 2003;60(2):184–9.
2. Furukawa TA, Kessler RC, Slade T, Andrews G. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychol Med 2003;33(2):357–62.
3. Andrews G, Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10). Aust N Z J Public Health 2001;25(6):494–7.
4. Brooks RT, Beard J, Steel Z. Factor structure and interpretation of the K10. Psychol Assess 2006;18(1):62–70.
5. Boswell JF, Kraus DR, Miller SD, Lambert MJ. Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychother Res 2013;(February 2014):37–41.
6. Stukenborg GJ, Blackhall L, Harrison J, Barclay JS, Dillon P, Davis MA, et al. Cancer patient-reported outcomes assessment using wireless touch screen tablet computers. Qual Life Res 2013; Epub ahead of print.
7. Zargaran E, Schuurman N, Nicol AJ, Matzopoulos R, Cinnamon J, Taulu T, et al. The electronic trauma health record: design and usability of a novel tabletbased tool for trauma care and injury surveillance in low resource settings. J Am Coll Surg 2014;218(1):41–50.
8. Batty MJ, Moldavsky M, Foroushani PS, Pass S, Marriott M, Sayal K, et al. Implementing routine outcome measures in child and adolescent mental health services: from present to future practice. Child Adolesc Ment Health 2013;18(2):82–7.
9. Bickman L. A measurement feedback system (MFS) is necessary to improve mental health outcomes. J Am Acad Child Adolesc Psychiatry 2008;47(10): 1114–9.
10. Miller SD, Duncan BL, Sorrell R, Brown GS. The partners for change outcome management system. J Clin Psychol 2005;61(2):199–208.
11. Furber G V., Crago AE, Meehan K, Sheppard TD, Hooper K, Abbot DT, et al. How Adolescents Use SMS (Short Message Service) to Micro-Coordinate Contact With Youth Mental Health Outreach Services. J Adolesc Heal 2010;48(1):113–5.
12. Using technology in practice: Case studies. Reachout Pro website. http://au.professionals.reachout.com/~/media/PDF/Professionals/Support%20workers/Part%203%20-%20Case%20Studies.ashx. Accessed January 24, 2014.
