Published on in Vol 7, No 5 (2019): May

Preprints (earlier versions) of this paper are available at, first published .
Technical Guidance for Clinicians Interested in Partnering With Engineers in Mobile Health Development and Evaluation

Technical Guidance for Clinicians Interested in Partnering With Engineers in Mobile Health Development and Evaluation

Technical Guidance for Clinicians Interested in Partnering With Engineers in Mobile Health Development and Evaluation


1Johns Hopkins University School of Medicine, Baltimore, MD, United States

2Johns Hopkins University Whiting School of Engineering, Baltimore, MD, United States

3David Geffen School of Medicine at University of California, Los Angeles, Los Angeles, CA, United States

4Johns Hopkins University School of Nursing, Baltimore, MD, United States

Corresponding Author:

Lochan M Shah, BA

Johns Hopkins University School of Medicine

733 N Broadway

Baltimore, MD, 21231

United States

Phone: 1 410 550 3350


Related ArticleThis is a corrected version. See correction statement in:

The explosion of mobile health (mHealth) interventions has prompted significant investment and exploration that has extended past industry into academia. Although research in this space is emerging, it focuses on the clinical and population level impact across different populations. To realize the full potential of mHealth, an intimate understanding of how mHealth is being used by patients and potential differences in usage between various demographic groups must also be prioritized. In this viewpoint, we use our experiences in building an mHealth intervention that incorporates an iOS app, Bluetooth-enabled blood pressure cuff, and Apple Watch to share knowledge on (1) how user interaction data can be tracked in the context of health care privacy laws, (2) what is required for effective, nuanced communication between clinicians and engineers to design mHealth interventions that are patient-centered and have high clinical impact, and (3) how to handle and set up a process to handle user interaction data efficiently.

JMIR Mhealth Uhealth 2019;7(5):e14124



Mobile health (mHealth) interventions promise to improve patient outcomes, increase self-management and health literacy, reduce health disparities, increase access to health services, and lower health care costs in ways previously unachievable [1,2]. However, much work is needed to move mHealth from the ivory tower of academia or the offices of startup companies to clinical practice, where mHealth interventions can be prescribed as adjuncts to therapy. To facilitate clinician and policy-maker decision making, researchers need to create an actionable knowledge base to identify the most effective, safe, and scalable interventions for improving individual and population health.

mHealth studies evaluating clinical and population-level impact have made strides in this direction. For example, BlueStar by WellDoc, a Food and Drug Administration (FDA) cleared mHealth intervention for patients with type 2 diabetes, demonstrated a significantly lower mean decrease in hemoglobin A1c over 1 year compared with the usual care group [3]. Similarly, a 2016 systematic review and meta-analysis found that across 27 studies, the mHealth group compared with the usual care group had increased adherence to medical therapy and ability to reach blood pressure and exercise goals [1]. However, other studies have found mixed results from mHealth interventions and have further concluded that the evidence for efficacy is still limited [4]. One concern that has been raised is the diversity of patient demographics and how this might lead to differential intervention usage and effectiveness. Indeed, studies have found that sex, age group, ethnicity, and family history significantly affect user engagement and that patients who are more engaged with a given mHealth intervention may be more likely to achieve the intended clinical outcomes [5-7].

Thus, to realize the full potential of mHealth interventions, validation of individual clinical outcomes is not enough. mHealth interventions are typically complex with multiple aims and components, including educational material, self-monitoring tools, and various other aspects of health behavior change constructs. Making meaningful insights and developing recommendations require understanding the causal model of how the intervention will achieve its intended benefit, how key components of the intervention interact with one another, and what combinations will optimize a return on investment [8]. Smartphones, in addition to functioning as a platform for such interventions, send data to a back-end system, where it can be leveraged to understand intervention use patterns in a way that avoids potential limitations of self-reported survey data such as recall bias. This kind of granular app usage analysis, while traditionally not associated with clinical research, is necessary for transitioning mHealth interventions toward real-world implementation and answering the question, “What works and for whom?”

Importantly, an analysis of mHealth back-end data presents important challenges, and few studies have examined clinical outcomes in the context of app usage data [8]. Here, we use our experiences in building the Corrie mHealth intervention as a framework to explore key questions that arise in constructing meaningful insights from back-end data. Corrie guides patients through recovery after acute myocardial infarction and incorporates an iOS app, a Bluetooth-enabled blood pressure cuff, and an Apple Watch. In addition to sharing nuances identified from our research and development process, we aimed to provide practical knowledge to help other clinicians work effectively with an engineering team to design mHealth interventions that are patient-centered and have high clinical impact.

The back-end data analysis allows mHealth teams to identify features that patients use most regularly and take that into account in an iterative design process, thus quantifying and supporting insights collected through patient surveys and focus groups. This enables teams to begin to understand if specific features may be associated with improved clinical outcomes and ultimately helps make interventions more patient-centered.

Although there are a number of different ways to track app usage [9], our intervention tracking is divided into 2 main categories: (1) the tracking of unique events and (2) the recording of time elapsed on view controllers. Such collection of intrasession measures (within sessions) of user engagement allow for the evaluation of specific user interactions and behaviors, while intersession measures (across sessions) evaluate long-term user engagement and satisfaction [9].

In the Corrie iOS app, unique app events are generated any time a patient (user) undertakes a specific action—for example, taking vitals, creating follow-up appointments, and completing learning goals. View controllers correspond to individual screens on iPhone. These view controllers can then be grouped by the feature they are a part of, allowing for the identification of features that patients spend the most time on (Figure 1). Commonly, nonhealth apps outsource this kind of analytics to third-party software development kits (SDKs) such as Firebase and Google Analytics. However, as many of those third-party software tools are not Health Insurance Portability and Accountability Act (HIPAA) compliant, mHealth teams may need to create their own HIPAA-compliant implementations, especially for detailed insights that can be correlated to patients and their clinical outcomes [10].

To make such meaningful insights, clinical teams must have continuous back-and-forth discussions with their engineering team to understand the context of data being obtained and the nuances of how it is captured. In addition to improving team efficiency, this detailed, continuous communication minimizes misinterpretation of the underlying data that could ultimately yield false conclusions. However, achieving this level of communication can be complicated by a technological language barrier and the inherent intricacies of back-end data collection.

Figure 1. An example of view controllers and their use in user interaction tracking.
View this figure

To address this, we found that it was critical for the clinical and engineering teams to collaborate more closely than is traditionally done. For example, our clinical team obtained the list of view controllers from our engineering team and classified them into 4 broad categories for feature usage analytics. We also walked through multiple use case scenarios with our engineering team to understand and fine-tune how data would be recorded in various clinical situations. This was particularly important for the development of a robust system to track unique app events. For example, we wanted to capture both time spent watching an educational video as well as completion of the video. If a patient went back to watch a video multiple times, we also wanted to capture that. However, if a patient accidentally pressed a medication taken button multiple times, we wanted that recorded as one event rather than multiple events (Figure 2).

Beyond recording app usage events, we also wanted to capture the values of clinical measurements such as heart rate, blood pressure, step count, weight, and mood to create rich datasets for a health outcome–based analysis. For these types of data, we emphasized the importance of querying and displaying the frequency of measurement and timestamps, as well as having well-defined data collection time windows. These details require a significant understanding of variable-capturing capabilities on the back-end but we believe they are key to building accurate and clinically relevant datasets. Ultimately, ensuring high quality and appropriately contextualized data will be critical as interest grows in applying machine learning–based analytical tools to generate actionable insights from large volumes of mHealth data, in the setting of both mHealth validation research as well as clinical scenarios using patient-generated data from commercial devices.

Similarly, when clinicians and engineers team up to solve a problem and work together as equal partners, the engineering team can gain an in-depth understanding of the clinical picture over time. Initially, many mHealth interventions start off with ad hoc combinations of the engineering team and interested clinical professionals partly serving as the data scientists [11]. For our team, this meant that when data were needed, the engineering team provided the relevant information to the clinical team. Close collaboration with the engineering team was required to sort through all the recorded variables and determine which were important to analyze and which were not. To facilitate that decision making, the engineering team needed to have a strong understanding of the clinical context. In addition to taking our engineering team through the discharge process and hosting our engineers on rounds in the hospital, we found it helpful to come together with our engineering team in clinically focused discussions by using real-time team messaging and collaboration platforms such as Slack (Slack Technologies, Inc). We have found this more efficient and effective than email and it provides an ongoing, continuous form of discussion outside the context of our regular face-to-face meetings and calls to facilitate clinical engineering collaboration. Slack discussions enable moment-to-moment understanding of clinical challenges, with the bonus of being able to answer urgent technical questions in real time. Importantly, this streamlined troubleshooting and prioritization of tasks for our engineering team as well, who were able to leverage real-time clinical perspective on potential features, content, and user workflow.

Figure 2. An example of data collection and product considerations when developing a new feature.
View this figure

Teams developing mHealth interventions must recognize that having the engineering team manually export back-end data is not a feasible long-term solution. As the intervention scales and the clinical team’s need for data extraction increases, the potential for severe bottlenecks warrants an early discussion about data management and access. For example, as our clinical team expanded, it became clear that identifying appropriate variables and packaging data into easy-to-read formats cost the engineering team valuable time that could have been better spent on other endeavors such as further developing the app.

In our case, this led to a stepwise progression of methods to access back-end data in a HIPAA-compliant manner. At first, our engineering team shared data via standard comma separated value (CSV) files over the HIPAA-compliant Johns Hopkins secure virtual desktop. We then transitioned to sharing a JavaScript Object Notation (JSON) file (a data-interchange format) and hiring a data scientist, giving clinicians access to all of the back-end data without the need for extensive teamwide technical training. Although we advocate hiring a dedicated data scientist to bridge the gap between clinicians and engineers early on, as studies scale up and move toward pragmatic trials that evaluate real-world implementation, we believe the end-goal also includes an HIPAA-compliant, queryable dashboard for clinicians, with data visualization and electronic medical record integration (Figure 3) Creating a system that allows clinical team members to intuitively access raw usage data not only decreases the burden on engineers but also empowers team members to better understand their data architecture and more confidently perform analyses based on their individual priorities.

Figure 3. Progression of user-interaction data management and extraction. CSV: comma separated value; JSON: JavaScript Object Notation.
View this figure

Going forward, it will be essential for research teams to evaluate mHealth interventions in the context of potentially differential app usage patterns and to create environments that support continuous, technically nuanced communication between their clinical and engineering counterparts. Although more business than clinical in nature, these changes to practice are essential to building interventions that patients actively want to use in real-world settings. In our experience with mHealth, being consumer-centered is synonymous with being patient-centered and is the future of health care delivery. We hope our experiences help interdisciplinary teams navigate similar challenges (Figure 4), as the capacity of mHealth interventions to revolutionize health care is directly linked to the quality of interventions developed through clinical-engineering partnership and studied through careful research.

Figure 4. Key takeaways. CSV: comma separated value; JSON: JavaScript Object Notation; EMR: electronic medical record.
View this figure

Authors' Contributions

The authors LS, WY, ES, RD, ML, DW, RS, FM, and SM provided substantial contributions to the conception and design of the work, contributed significantly to drafts of the work, and critically revised it for key intellectual content. Additionally, final approval of the version to be published was solicited from each, and each agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work were appropriately investigated and resolved. Additionally, the author LS took primary responsibility for communication with the journal during the manuscript submission, peer review, and publication process.

Conflicts of Interest

There was no specific funding for this paper. EMS has received the following financial support for research, authorship, and/or publication: NIH/NINR F31 NR017328, Ruth L Kirschstein National Research Service Award and NIH/NINR T32 NR012704, Pre-Doctoral Fellowship in Interdisciplinary Cardiovascular Health Research. The sponsor had no role in study design, collection, analysis, and interpretation of data, writing the report, and the decision to submit the report for publication. SSM reports receiving grants and research support from the PJ Schafer Cardiovascular Research Fund, the David and June Trone Family Foundation, American Heart Association, Aetna Foundation, Maryland Innovation Initiative, Nokia, iHealth, Google, and Apple, as well as receiving personal fees for serving on scientific advisory boards for Amgen, Sanofi/Regeneron, Quest Diagnostics, Esperion, Novo Nordisk, Akcea Therapeutics. In addition, SSM reports having patent applications pending. FAM and SSM are founders of and hold equity in Corrie Health, which intends to further develop the platform. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflict of interest policies. Furthermore, the Corrie Health Digital Platform has received material support from Apple and iHealth, and funding from the Maryland Innovation Initiative, Wallace H Coulter Translational Research Partnership, Louis B Thalheimer Fund, and JH Individualized Health Initiative. Corrie Health, as described in this work, was developed by FAM, MAL, and SSM. They are also founders of and hold equity in Corrie Health, which intends to further develop the digital platform. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflict of interest policies.

  1. Gandhi S, Chen S, Hong L, Sun K, Gong E, Li C, et al. Effect of mobile health interventions on the secondary prevention of cardiovascular disease: systematic review and meta-analysis. Can J Cardiol 2017 Feb;33(2):219-231. [CrossRef]
  2. Beatty A, Fukuoka Y, Whooley M. Using mobile technology for cardiac rehabilitation: a review and framework for development and evaluation. J Am Heart Assoc 2013 Nov 01;2(6):e000568 [FREE Full text] [CrossRef] [Medline]
  3. Quinn C, Shardell M, Terrin M, Barr E, Ballew S, Gruber-Baldini A. Cluster-randomized trial of a mobile phone personalized behavioral intervention for blood glucose control. Diabetes Care 2011 Sep;34(9):1934-1942 [FREE Full text] [CrossRef] [Medline]
  4. Marcolino M, Oliveira J, D'Agostino M, Ribeiro A, Alkmim M, Novillo-Ortiz D. The impact of mHealth interventions: systematic review of systematic reviews. JMIR Mhealth Uhealth 2018 Jan 17;6(1):e23 [FREE Full text] [CrossRef] [Medline]
  5. Bell A, Fonda S, Walker M, Schmidt V, Vigersky R. Mobile phone-based video messages for diabetes self-care support. J Diabetes Sci Technol 2012 Mar 01;6(2):310-319 [FREE Full text] [CrossRef] [Medline]
  6. Lin P, Grambow S, Intille S, Gallis J, Lazenka T, Bosworth H, et al. The association between engagement and weight loss through personal coaching and cell phone interventions in young adults: randomized controlled trial. JMIR Mhealth Uhealth 2018 Oct 18;6(10):e10471 [FREE Full text] [CrossRef] [Medline]
  7. Goyal S, Morita P, Picton P, Seto E, Zbib A, Cafazzo J. Uptake of a consumer-focused mHealth application for the assessment and prevention of heart disease: the <30 Days study. JMIR Mhealth Uhealth 2016 Mar 24;4(1):e32 [FREE Full text] [CrossRef] [Medline]
  8. Heffner J, Vilardaga R, Mercer L, Kientz J, Bricker J. Feature-level analysis of a novel smartphone application for smoking cessation. Am J Drug Alcohol Abuse 2015 Jan;41(1):68-73 [FREE Full text] [CrossRef] [Medline]
  9. Lalmas M, O'Brien H, Yom-Tov E. Measuring user engagement: Synthesis Lectures on Information Concepts, Retrieval, and Services. San Rafael, California: Morgan & Claypool Publishers; Nov 2014:1-132.
  10. 2019. Google Analytics Help. Best practices to avoid sending Personally Identifiable Information (PII)   URL: [WebCite Cache]
  11. Huesch M, Mosher T. NEJM Catalyst. Using It or Losing It? The Case for Data Scientists Inside Health Care   URL: [accessed 2019-03-27] [WebCite Cache]

CSV: comma separated value
FDA: Food and Drug Administration
HIPAA: Health Insurance Portability and Accountability Act
JSON: JavaScript Object Notation
mHealth: mobile health
SDK: software development kit

Edited by G Eysenbach; submitted 27.03.19; peer-reviewed by S Kakarmath, A Hussain; accepted 23.04.19; published 15.05.19


©Lochan M Shah, William E Yang, Ryan C Demo, Matthias A Lee, Daniel Weng, Rongzi Shan, Shannon Wongvibulsin, Erin M Spaulding, Francoise A Marvel, Seth S Martin. Originally published in JMIR mHealth and uHealth (, 15.05.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.