Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps

John Torous, MD MBI @JohnTorousMD

@erin_michalak @jenmnicholas1 The actual metrics are not well defined and often misused in this space. More complex… https://t.co/hCtYjOSeB1

2:26 AM · Aug 21, 2018

5

2

Erin Michalak @erin_michalak

huh, thank you John, so interesting. I predict this project will be like our previous ones - we need the standardiz… https://t.co/4g8RAeqnvn

2:39 AM · Aug 21, 2018

0

0

Skye Barbic she/her/hers @skye_barbic

RT @JohnTorousMD: @erin_michalak @jenmnicholas1 The actual metrics are not well defined and often misused in this space. More complex metri…

4:43 AM · Aug 21, 2018

5

2

Raymond Lam @DrRaymondLam

RT @JohnTorousMD: @erin_michalak @jenmnicholas1 The actual metrics are not well defined and often misused in this space. More complex metri…

3:17 PM · Aug 21, 2018

5

2

John Torous, MD MBI @JohnTorousMD

Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https://t.co/jlrpzizokC

3:25 PM · Feb 10, 2016

21

22

Dr Bobby Lazzara @DrBobbyIsIn

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:26 PM · Feb 10, 2016

21

22

Storydoc.ai @storydoc_ai

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:32 PM · Feb 10, 2016

21

22

Steven Chan, MD MBA @StevenChanMD

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:32 PM · Feb 10, 2016

21

22

Payer+Provider @PayerProvider

Can mHealth app ratings be trusted? See https://t.co/LWCjQnNKsg

3:33 PM · Feb 10, 2016

0

0

MindBamboo @MindBamboo

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:33 PM · Feb 10, 2016

21

22

JMIR Publications @jmirpub

JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking C... https://t.co/IyrbtzoKSe

3:33 PM · Feb 10, 2016

5

5

Dr Ashish Dhawad @docdhawad

RT JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depressi… https://t.co/8EwCybykzl

3:42 PM · Feb 10, 2016

0

0

DavisLiuMD ???? @davisliumd

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:45 PM · Feb 10, 2016

21

22

Alison Darcy @alisonmdarcy

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

4:29 PM · Feb 10, 2016

21

22

David K Ahern @dahern1

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

5:06 PM · Feb 10, 2016

21

22

BrighamResearch @BrighamResearch

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

5:24 PM · Feb 10, 2016

21

22

Molly Waring @DrMollyWaring

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

6:40 PM · Feb 10, 2016

21

22

Adam Landman @landmaad

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

7:19 PM · Feb 10, 2016

21

22

Arshya Vahabzadeh MD @VahabzadehMD

Disagreement rife on how to rate #mhealth apps -paper-> https://t.co/uF5HLWz2Ug cc @JohnTorousMD @StevenChanMD @APAPsychiatric

7:52 PM · Feb 10, 2016

8

4

John Torous, MD MBI @JohnTorousMD

RT @VahabzadehMD: Disagreement rife on how to rate #mhealth apps -paper-> https://t.co/uF5HLWz2Ug cc @JohnTorousMD @StevenChanMD @APAPsychi…

7:58 PM · Feb 10, 2016

8

4

MindBamboo @MindBamboo

RT @VahabzadehMD: Disagreement rife on how to rate #mhealth apps -paper-> https://t.co/uF5HLWz2Ug cc @JohnTorousMD @StevenChanMD @APAPsychi…

8:21 PM · Feb 10, 2016

8

4

Timothy Aungst @TDAungst

RT @VahabzadehMD: Disagreement rife on how to rate #mhealth apps -paper-> https://t.co/uF5HLWz2Ug cc @JohnTorousMD @StevenChanMD @APAPsychi…

8:34 PM · Feb 10, 2016

8

4

Atul Butte @atulbutte

Turns out even #mHealth experts have a hard time consistently rating medical apps! https://t.co/vIep3UgdUk HT @JohnTorousMD

8:49 PM · Feb 10, 2016

10

4

Dr Ashish Dhawad @docdhawad

RT atulbutte: Turns out even #mHealth experts have a hard time consistently rating medical apps! https://t.co/udhulVwV5h  HT JohnTorousMD

8:50 PM · Feb 10, 2016

0

0

Ask Apps Developers @AskAppDev

RT @atulbutte: Turns out even #mHealth experts have a hard time consistently rating medical apps! https://t.co/vIep3UgdUk HT @JohnTorousMD

8:53 PM · Feb 10, 2016

10

4

Dena Mendelsohn @HealthEDena

RT @atulbutte: Turns out even #mHealth experts have a hard time consistently rating medical apps! https://t.co/vIep3UgdUk HT @JohnTorousMD

9:23 PM · Feb 10, 2016

10

4

Avinesh Bhar, MD MBA @AviBhar

RT @atulbutte: Turns out even #mHealth experts have a hard time consistently rating medical apps! https://t.co/vIep3UgdUk HT @JohnTorousMD

10:26 PM · Feb 10, 2016

10

4

Mike Clark @clarkmike

Study: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression & Smoking Cessation Apps - https://t.co/UFTEsyj918

11:00 PM · Feb 10, 2016

1

0

Adam Miner @electrotipe

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

12:49 AM · Feb 11, 2016

21

22

Sandeep Pulim @SPulim

RT @JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking C... http…

3:20 AM · Feb 11, 2016

5

5

Physical-Mental Health Manchester @PhysMentManc

@JohnTorousMD and Harvard mHealth team create new, valid rating measures for apps https://t.co/doK5nUc2yz @NewMindNetwork @JMedInternetRes

10:01 AM · Feb 11, 2016

2

5

Semeoticons @Semeoticons

RT @atulbutte "Turns out even #mHealth experts have a hard time consistently rating medical apps! " https://t.co/CCOYpPthqx

12:45 PM · Feb 11, 2016

0

0

Henrique Gomide @hpgomide

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

1:14 PM · Feb 11, 2016

21

22

JMIR Publications @jmirpub

RT @joefirth7: @JohnTorousMD and Harvard mHealth team create new, valid rating measures for apps https://t.co/doK5nUc2yz @NewMindNetwork @J…

2:15 PM · Feb 11, 2016

2

5

Michael Dillhyon @SwissGator

RT @atulbutte: Turns out even #mHealth experts have a hard time consistently rating medical apps! https://t.co/vIep3UgdUk HT @JohnTorousMD

2:21 PM · Feb 11, 2016

10

4

Danielle Siarri @innonurse

RT @joefirth7: @JohnTorousMD and Harvard mHealth team create new, valid rating measures for apps https://t.co/doK5nUc2yz @NewMindNetwork @J…

2:22 PM · Feb 11, 2016

2

5

Dr Nikki Newhouse @nikkinewhouse

RT @joefirth7: @JohnTorousMD and Harvard mHealth team create new, valid rating measures for apps https://t.co/doK5nUc2yz @NewMindNetwork @J…

2:58 PM · Feb 11, 2016

2

5

Vanessa Mason @vanessamason

RT @joefirth7: @JohnTorousMD and Harvard mHealth team create new, valid rating measures for apps https://t.co/doK5nUc2yz @NewMindNetwork @J…

3:18 PM · Feb 11, 2016

2

5

Andy Oram @praxagora

Worth reading for anyone concerned with quality and efficacy of health-related apps - evaluations are hard to trust https://t.co/n2qmcrBGWp

6:58 PM · Feb 11, 2016

0

0

Aviv Shachak ???????????????? @AvivShachak

RT @joefirth7: @JohnTorousMD and Harvard mHealth team create new, valid rating measures for apps https://t.co/doK5nUc2yz @NewMindNetwork @J…

9:12 PM · Feb 11, 2016

2

5

Payer+Provider @PayerProvider

RT @JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking C... http…

10:57 PM · Feb 11, 2016

5

5

Dr. Adam Powell @adamcpowell

RT @JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking C... http…

10:58 PM · Feb 11, 2016

5

5

Dr Ashish Dhawad @docdhawad

RT JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depressi… https://t.co/8EwCybykzl

11:34 PM · Feb 11, 2016

0

0

JMIR Publications @jmirpub

Quality control needed for health apps: Brigham researchers https://t.co/LEJSODq19p about @JohnTorousMD JMIR study https://t.co/xJkrBwfP1h

9:38 PM · Feb 12, 2016

5

7

Claudia Pagliari @EeHRN

RT @JMedInternetRes: Quality control needed for health apps: Brigham researchers https://t.co/LEJSODq19p about @JohnTorousMD JMIR study htt…

9:41 PM · Feb 12, 2016

5

7

Dr. Cynthia Graham @cygraham_graham

RT @JMedInternetRes: Quality control needed for health apps: Brigham researchers https://t.co/LEJSODq19p about @JohnTorousMD JMIR study htt…

9:42 PM · Feb 12, 2016

5

7

John Torous, MD MBI @JohnTorousMD

RT @JMedInternetRes: Quality control needed for health apps: Brigham researchers https://t.co/LEJSODq19p about @JohnTorousMD JMIR study htt…

9:52 PM · Feb 12, 2016

5

7

robyn whittaker @rawegd

RT @JMedInternetRes: Quality control needed for health apps: Brigham researchers https://t.co/LEJSODq19p about @JohnTorousMD JMIR study htt…

10:11 PM · Feb 12, 2016

5

7

Steven Willey, MD @DrSteve

Hard to evaluate #healthapps says @JMedInternetRes study. Physicians should focus only on medically-proven apps https://t.co/pOCO80yhU4

11:17 PM · Feb 12, 2016

1

0

Burza Burzyńska @JoanBurzynska

RT @JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking C... http…

6:05 AM · Feb 13, 2016

5

5

Dr Ashish Dhawad @docdhawad

RT JMedInternetRes: JMIR #mHealth: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depressi… https://t.co/8EwCybykzl

6:07 AM · Feb 13, 2016

1

0

Dr Ashish Dhawad @docdhawad

RT doctick: #mhealth "Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smokin… https://t.co/zpVgaSJknx

11:07 AM · Feb 13, 2016

0

0

ORCHA @OrchaHealth

Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps https://t.co/Vgeb9V8Q2B

12:50 PM · Feb 13, 2016

2

2

UKMedLibs @UKMedLibs

Good timing re #ukmedlibs chat “@JMedInternetRes: Quality control needed for health apps: @JohnTorousMD JMIR study https://t.co/RKwqT1XMVj”

2:46 PM · Feb 13, 2016

4

2

Jennifer M. Joe, MD @JenniferJoeMD

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:22 PM · Feb 13, 2016

21

22

Julio Mayol @juliomayol

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:23 PM · Feb 13, 2016

21

22

John Torous, MD MBI @JohnTorousMD

RT @UKMedLibs: Good timing re #ukmedlibs chat “@JMedInternetRes: Quality control needed for health apps: @JohnTorousMD JMIR study https://…

4:27 PM · Feb 13, 2016

4

2

Dustin at Distance @DU5TB1N

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

6:13 PM · Feb 13, 2016

21

22

Teen Whisperer @EliseBerlanMD

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

2:23 AM · Feb 15, 2016

21

22

Mike Tyldesley @mike_nwcahsn

RT @OrchaUK: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps https://t.co/Vgeb…

2:33 PM · Feb 15, 2016

2

2

Urban Design Lab @urbanclimate

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

3:20 PM · Feb 15, 2016

21

22

Julia Reynolds @JulsReynolds

RT @OrchaUK: Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps https://t.co/Vgeb…

3:27 PM · Feb 16, 2016

2

2

Brian Eastwood @Brian_Eastwood

Surprise! #mHealth app ratings aren't very helpful, via @JMedInternetRes https://t.co/1xQFvIbqiY --> Providers must vet apps themselves

5:28 PM · Feb 16, 2016

1

4

Dr Ashish Dhawad @docdhawad

RT Brian_Eastwood: Surprise! #mHealth app ratings aren't very helpful, via JMedInternetRes https://t.co/w5U7hK3Ucr --> Providers must vet …

5:28 PM · Feb 16, 2016

0

0

Laurie Gordon M.D. @DrLaurieG

RT @Brian_Eastwood: Surprise! #mHealth app ratings aren't very helpful, via @JMedInternetRes https://t.co/1xQFvIbqiY --> Providers must vet…

5:32 PM · Feb 16, 2016

1

4

The Doctors Company @doctorscompany

RT @Brian_Eastwood: Surprise! #mHealth app ratings aren't very helpful, via @JMedInternetRes https://t.co/1xQFvIbqiY --> Providers must vet…

5:33 PM · Feb 16, 2016

1

4

Cinema Olografico @holocinema

RT davidbychkov: Brian_Eastwood: Surprise! #mHealth app ratings aren't very helpful, via JMedInternetRes https://t.co/PEgsm4Y7k5 --> Provi…

5:34 PM · Feb 16, 2016

0

0

healthnotes @healthnotesco

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

6:25 PM · Feb 21, 2016

21

22

MedGizmo @MedGizmo

quality review of #healthcare #apps https://t.co/OaDdl8ZdrI #mHealth #mHealthApps via #JMIR

4:56 AM · Feb 27, 2016

3

1

Dr Ashish Dhawad @docdhawad

RT MedGizmo: quality review of #healthcare #apps https://t.co/MGm3DOlQb6  #mHealth #mHealthApps via #JMIR

5:02 AM · Feb 27, 2016

1

0

Medlium @Medlium

RT @MedGizmo: quality review of #healthcare #apps https://t.co/OaDdl8ZdrI #mHealth #mHealthApps via #JMIR

5:56 AM · Feb 27, 2016

3

1

MIT Hacking Medicine @mithackmed

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

5:26 PM · Feb 28, 2016

21

22

Jorge A Rodriguez MD @translatedmed

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

5:36 PM · Feb 28, 2016

21

22

マーキュリー２世 @uranus_2

メンタルヘルスアプリの評価に関する評定者間信頼性。Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and …https://t.co/j90NTVFMCF

7:32 AM · Feb 29, 2016

3

0

Sheraz Khan @wordsworkout

RT @UKMedLibs: Good timing re #ukmedlibs chat “@JMedInternetRes: Quality control needed for health apps: @JohnTorousMD JMIR study https://…

9:32 PM · Feb 29, 2016

4

2

Sheraz Khan @wordsworkout

RT @JMedInternetRes: Quality control needed for health apps: Brigham researchers https://t.co/LEJSODq19p about @JohnTorousMD JMIR study htt…

9:33 PM · Feb 29, 2016

5

7

Sheraz Khan @wordsworkout

RT @VahabzadehMD: Disagreement rife on how to rate #mhealth apps -paper-> https://t.co/uF5HLWz2Ug cc @JohnTorousMD @StevenChanMD @APAPsychi…

9:33 PM · Feb 29, 2016

8

4

Sheraz Khan @wordsworkout

RT @JohnTorousMD: Our team's new paper on #mhealth #mentalhealth app ratings challenges in @JMedInternetRes https://t.co/E8VxJXltpA https…

9:33 PM · Feb 29, 2016

21

22

jenni beattie @jennibeattie

Analysis of Top Depression & Smoking Cess #Apps https://t.co/VDoPeVQsDF #digitalhealth #mhealth https://t.co/PdkVhiYj9b

8:25 PM · Mar 31, 2016

3

1

JMIR Publications @jmirpub

RT @jennibeattie: Analysis of Top Depression & Smoking Cess #Apps https://t.co/VDoPeVQsDF #digitalhealth #mhealth https://t.co/PdkVhiYj9b

9:52 PM · Apr 01, 2016

3

1

Alejandra Fernández @AleFer_endo

Interrater Reliability of #mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation #Apps. https://t.co/U1ioWRQRWw

8:10 PM · Apr 12, 2016

0

0

Eva Aumatell ????️ @eaumatell

Interrater Reliability of #mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps https://t.co/NSXV3YtxTj

11:31 AM · May 27, 2016

1

0

Lisa Newmark @lpnewmark

RT @Brian_Eastwood: Surprise! #mHealth app ratings aren't very helpful, via @JMedInternetRes https://t.co/1xQFvIbqiY --> Providers must vet…

2:59 PM · Dec 15, 2016

1

4