This paper is in the following e-collection/theme issue:

Ecological Momentary Assessment (EMA) (322) Research Instruments, Questionnaires, and Tools (591) Design, Usability, and Evaluation of Research Instruments, Scales, and Measures (56) Depression and Mood Disorders; Suicide Prevention (1393)

Published on in Vol 8, No 7 (2020): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/17926, first published .
Psychometric Properties of the Korean Version of the PsyMate Scale Using a Smartphone App: Ecological Momentary Assessment Study

Psychometric Properties of the Korean Version of the PsyMate Scale Using a Smartphone App: Ecological Momentary Assessment Study

Psychometric Properties of the Korean Version of the PsyMate Scale Using a Smartphone App: Ecological Momentary Assessment Study

Authors of this article:

Yong Sook Yang1 Author Orcid Image ;   Gi Wook Ryu1 Author Orcid Image ;   Philippe A E G Delespaul2 Author Orcid Image ;   Mona Choi1 Author Orcid Image
Article Authors Cited by (1) Tweetations (1) Metrics

    Original Paper

    1Mo-Im Kim Nursing Research Institute, Yonsei University College of Nursing, Seoul, Republic of Korea

    2Department of Psychiatry and Neuropsychology, Faculty of Health Medicine and Lifesciences, Maastricht University, Maastricht, Netherlands

    Corresponding Author:

    Mona Choi, PhD

    Mo-Im Kim Nursing Research Institute

    Yonsei University College of Nursing

    50 Yonsei-ro, Seodaemun-gu

    Seoul, 03722

    Republic of Korea

    Phone: 82 2 2228 3341

    Email: monachoi@yuhs.ac


    Background: Ecological momentary assessment (EMA) is a method for capturing the changes in the variables in daily life with increased accuracy and decreased recall bias. The PsyMate scale assesses momentary moods in daily life and can be used in various settings.

    Objective: The aim of this study was to develop a Korean version of the PsyMate (K-PsyMate) scale and evaluate its psychometric properties by using the EMA method in patients with moyamoya disease (MMD) in South Korea.

    Methods: Patients with MMD aged over 18 years were recruited from July 2018 to January 2019 at the inpatient and outpatient departments of a university hospital in South Korea. The K-PsyMate scale comprising 13 items was developed following a translation/back translation approach of the English version and loaded onto a mobile app. Participants were instructed to enter their moods 4 times a day for 7 consecutive days. Content validity index, factor analysis, and Pearson’s correlation were performed for validity analysis. For reliability, intraclass correlation coefficients between the first and last measurements were estimated by mean rating, absolute agreement, and a 2-way mixed-effects model. Usability was analyzed through a descriptive analysis, 2-tailed t test, and analysis of variance, and the results were confirmed by Mann-Whitney U test and the Kruskal-Wallis test, as the dependent variable was not normally distributed.

    Results: In total, 1929 assessments from 93 patients were analyzed. The mean age of the participants was 40.59 (SD 10.06) years, and 66 (71%) of the 93 participants were women. Content validity was excellent as content validity index was 0.99, and 2 factors, negative affect and positive affect, were derived by an exploratory factor analysis. The correlations between the subdomains of the K-PsyMate scale and Hospital Anxiety and Depression Scale were significant (P<.001). The agreement between the first and last measurements was poor to moderate according to the obtained intraclass correlation coefficient values. Usability was evaluated by 67 (72%) out of the 93 participants. The participants rated the accuracy of assessing their momentary moods on the app at 4.13 (SD 0.97), easiness in understanding questions, operating, and inputting answers at 4.12 (SD 0.88), and interruption by the survey alarms at 2.48 (SD 1.02) out of 5.

    Conclusions: The K-PsyMate scale has good validity but poor to moderate agreement, which reflects the characteristics of the EMA data collected in real and natural living environments without control. The findings of our study show that the K-PsyMate scale uploaded in a mobile app can be a valid and reliable tool for evaluating the momentary mood of patients with MMD because using a mobile app is convenient and patients are familiar with their own smartphones, which they use in their daily lives.

    JMIR Mhealth Uhealth 2020;8(7):e17926

    doi:10.2196/17926

    Keywords

    psychometrics; reliability; validity; PsyMate scale; ecological momentary assessment 


    Moyamoya disease (MMD), also called as spontaneous occlusion of the circle of Willis, is a chronic progressive cerebrovascular disease with no known cause and is characterized by fine collateral vessel networks that resemble hazy cloud-like puffs of smoke in the brain [1,2]. MMD occurs worldwide with high prevalence in East Asian countries such as Japan and Korea [3]. Unlike MMD in children, MMD in adults is associated with potential rupture or ischemia of the blood vessels in the brain, similar to that reported in other cerebrovascular diseases [4,5], and this can lead to high mortality [6]. The clinical features of patients with MMD often appear as mental deterioration, headache, and speech impairment [6], which are affected by mood changes. Adults with MMD also show high levels of anxiety and depression [7] and their moods change over time during the course of life [8]. Negative mood and mood changes in adults with MMD may affect their motivation for proper health behaviors and eventually affect the health outcomes of patients with similar cerebrovascular diseases [7]. Therefore, monitoring the moods of adults with MMD can be critical for both patients and health care personnel.

    Measuring moods may help people better understand their emotional states. The measuring tool to self-assess moods should be understandable and comfortable to use; at the same time, the tool should be able to assess the mood accurately and capture the fluctuations and changes in the mood reliably since mood varies according to context and time [9,10]. To avoid mood changes caused by the tool itself, the participants should be allowed to respond to the questionnaire in a flexible and friendly atmosphere of their choice.

    Traditionally, mood has been assessed by retrospective measures, including pen and paper tests [11], but these methods lead to recall bias and risk of reduced accuracy [9,10]. However, momentary assessments such as ecological momentary assessment (EMA) or experience sampling methods allow increased accuracy and reduced bias because these assessments are performed in real-life environments and through repeated assessments at multiple times [9-11]. The EMA method is known for its effectiveness in capturing changes [12] and understanding daily changes in psychological states such as mood and stress [13,14]. It is broadly used to assess the psychological aspects of the participants with [15-17] and without mental health problems [18]. The increased prevalence of smartphones and advanced mobile technology has made people more familiar with the use of smartphones in their daily lives, and EMA methods using smartphones are available [19-22]. The prevalence of smartphone use in Korean adults is about 95% [23], and about 96.4% of the population under the age of 70 years in Korea can access internet services [24]. In a study conducted with healthy participants in Korea, the EMA method was evaluated as comfortable to use and easy to understand when the mood was assessed using a mobile app installed in their own smartphones [22].

    Various measurement tools using mobile technology have been developed to measure moods in daily life. The PsyMate scale was developed to assess momentary moods relating to the daily life of the participants [25], and it has been used in various settings and populations [26,27]. In this study, we aimed to develop a Korean version of the PsyMate (K-PsyMate) scale, upload it on a mobile app to measure the momentary moods of Korean adults with MMD, and examine the psychometric properties of this scale.


    Participants

    Patients with MMD diagnosed via magnetic resonance angiography and aged over 18 years were recruited from July 2018 to January 2019 at the outpatient department or the inpatient wards of a university hospital in Seoul, South Korea. Patients who could not communicate owing to neurological problems were excluded by a physician and the physician’s assistant who were involved in the treatment of patients with MMD. The Korean version of the Mini-Mental State Examination [28] was used to assess the cognitive impairment of the participants after acquiring the permission to use it from the provider. Participants with a score under 24 in the Korean version of the Mini-Mental State Examination were excluded. Participants who used operating systems other than the Android system were also excluded, since the developed mobile app was available only for the Android operating system as described in the previous study [22]. Informed consent was obtained from all the participants when they enrolled in the study. This study was approved by the Institutional Review Board of the Yonsei University Health System (No. 4-2018-0385).

    Measurements

    Participants’ Characteristics and Baseline Mood

    All participants were asked to provide their demographic and disease-specific information at the baseline measurement through a questionnaire that was developed by our research team. Demographics included age, gender, marital status, people living with, job status, monthly household income, education level, and smoking and drinking history. Detailed information on the participants’ demographics has been presented in our previous work [29]. Regarding the baseline mood, the Hospital Anxiety and Depression Scale (HADS) [30] was administered after obtaining permission to use it. The HADS is known as a globally reliable and valid scale for assessing mental health in hospital settings [31]. It consists of 14 items, 7 for anxiety and 7 for depression, and it is measured on a 4-point scale ranging from 0 to 3. A higher value indicates a higher level of anxiety and depression. In this study, we administered the Korean version of HADS [32], for which Cronbach α was .89 and .86 for anxiety and depression subscales, respectively [32]. The HADS was assessed to compare the momentary negative affect and positive affect that were measured by the K-PsyMate scale, as described in a previous study [25].

    K-PsyMate Scale for Measuring Momentary Mood

    The mood scale of the K-PsyMate was used for the EMA section of this study. This scale consists of 13 items, of which 4 items measure positive affect and 9 items assess negative affect. The positive affect comprised feeling “cheerful,” “satisfied,” “relaxed,” and “globally well,” and the negative affect comprised feeling “lonely,” “guilty,” “worried,” “down,” “threatened,” “insecure,” “irritated,” “frightened,” and “suspicious.” These items were rated on a 7-point scale, with “1” indicating “not at all” and “7” indicating “very much.” Momentary mood was measured while considering the environmental context, taking into account what the participants were doing, where they were, and with whom they were when responding to the questionnaire.

    Usability Measure

    The usability of the EMA method on the mobile phone was evaluated by the participants at the end of the survey. A 10-item questionnaire was uploaded in the app: 2 items for accuracy, 3 items for easiness, 2 items for enjoyment, 2 items for intention, and 1 item for interruption. It was adapted from a previous study [33] with the permission from the developer and scored on a 5-point scale (1=strongly disagree to 5=strongly agree). An open-ended question was included for the participants to offer any additional comment or opinion. The Cronbach α of the scale in this study was .74.

    Procedure

    Translation of the PsyMate

    We freely downloaded the English version of the PsyMate scale [34] with permission from the copyright holder. To develop the K-PsyMate scale, we used the translation/back translation approach. We employed a committee approach for translation, in which we consulted 3 nursing professors who were fluent in both English and Korean to translate the English version of PsyMate [25]. Through this method, we identified language and cultural differences in selecting words. The translated PsyMate items were prepared for back translation during which we consulted 3 different nursing professors who were fluent in both English and Korean. The research team compared the back-translated PsyMate with the original English version to confirm that the translated Korean version was accurate. After confirming the translated Korean version through back translation, the research team finalized the K-PsyMate draft. This draft was then evaluated by a linguist of the Korean language prior to determining the final version. Eventually, the K-PsyMate version was finalized by the research team.

    Data Collection

    While obtaining informed consent from each patient, members of the research team obtained baseline data on demographic and disease-specific information by using the structured questionnaire and on mood status by using the HADS. They helped each participant install the mobile app on their mobile phone and register themselves, and they provided time for practice. Participants selected the EMA sampling period to assess momentary moods according to their convenience. They were guided by an alarm to respond 4 times a day for 7 consecutive days (4 times × 7 days = 28 times/person) at 60-minute blocked semirandom intervals. Notifications were sent in the morning between 8 AM and 9 AM, early afternoon between noon and 1 pm, evening between 5 PM and 6 PM, and at night between 9 PM and 10 PM. A reminder for each scheduled measurement was sent to the participants who did not input the response within 45 minutes after the first notification. The contact numbers and the email addresses of the members of the research team were given to the participants to contact the team if they had any questions or if they faced any difficulties during the assessment period. Finally, patients were requested to complete a 10-item survey at the end of the EMA session on the usability of the EMA method using a mobile app, including an open-ended question.

    Statistical Analysis

    Data were analyzed using the SPSS v.25.0 (IBM Corp). We used descriptive statistics to analyze the baseline data of the participants’ characteristics, the HADS scores, and the positive affect and negative affect of the K-PsyMate scale measured by the EMA method. We applied a 2-tailed t test and analysis of variance to examine the differences between groups according to the general characteristics. Regarding validity tests, we used the content validity index (CVI), which calculated scale-level CVI (S-CVI) and each item-level CVI (I-CVI). To assess CVI, each item was appraised by an expert panel consisting of 6 nursing professors. The CVI for each item was scored using a 4-point scale, where 1=not relevant, 2=somewhat relevant, 3=quite relevant, and 4=highly relevant. S-CVI and I-CVI were evaluated with the cut-off of S-CVI level at 0.90 and I-CVI at 0.78 [35]. We performed exploratory factor analysis (EFA) to examine construct validity using principal component analysis and varimax rotation. We also used Pearson’s correlation (r) to assess the association between the HADS and PsyMate in the previous study [25]. We calculated intraclass correlation coefficients (ICCs) for evaluating the test-retest reliability of the measurement tool. Reliability reflects both the degree of correlation and agreement between the measurements, and ICCs are an index that show the degree of correlation and agreement between the measurements [36]. Among the 10 different forms of ICC [37,38], we calculated the ICCs of each item between the first and the last measurement by mean rating, absolute agreement, and a 2-way mixed-effects model, since we tested absolute agreement of the mean of multiple measurements from a nonrandomly selected population [36]. The obtained ICC values of each item were interpreted by its 95% CI ranges [36]. The ICC estimates in the 95% CI were evaluated by considering ICC<0.50, 0.50≤ICC<0.75, 0.75≤ICC<0.90, and ICC≥0.90 as poor, moderate, good, and excellent reliability, respectively [39]. For usability evaluation, we performed descriptive analysis, 2-tailed t test, and analysis of variance, and we confirmed the results by Mann-Whitney U test and the Kruskal-Wallis test, as the dependent variable was not normally distributed.


    Participants

    We recruited patients with MMD from the inpatient and outpatient departments of a tertiary hospital in South Korea. We approached 119 patients of which 109 (91.6%) agreed to participate, and they were enrolled using the mobile app. Among the 109 patients, 98 (89.9%) participated in the EMA session. The data of 93 participants who provided more than 3 measurements considered as EMA data [40] and who provided 1929 responses (74.1%) out of the 2604 scheduled assessments were analyzed. The mean age of the participants was 40.59 (SD 10.06) years, and 66 of the 93 participants (71%) were women. The HADS anxiety and HADS depression scores were 7.17 (SD 3.38) and 7.14 (SD 3.51), respectively. The overall mean negative affect and positive affect scores were 2.15 (SD 1.12) and 4.70 (SD 1.31) out of 7, respectively.

    Validity of the K-PsyMate Scale

    We calculated S-CVI and I-CVI as evaluated by the expert panel. The S-CVI was 0.99 (I-CVI range 0.83-1.00), which represented an excellent level of content validity. We also calculated the S-CVIs of 2 factors of negative affect and positive affect according to the original study [25], and they were 1.00 and 0.96, respectively. The I-CVIs are presented in Table 1.

    EFA was performed based on 1929 completed assessments from 93 participants. We confirmed the presence of latent factors in the observed items with excellent Kaiser-Meyer-Olkin of 0.93 and with significant Bartlett’s sphericity measures (P<.05). Two factors were derived with the eigenvalue greater than 1 with a cumulative percentage of explained variance of 71.1%. All items were assigned to either the negative affect (Factor 1) or positive affect (Factor 2) factor. These 2 factors were negatively related (r=–0.67). The results of the EFA are shown in Table 2.

    The subdomains of negative affect and positive affect of the K-PsyMate scale and the HADS anxiety and depression were significantly correlated with coefficients (r) ranging between 0.33 and 0.68. The Cronbach α of the HADS anxiety and depression in this study were .84 and .79, respectively. The correlation coefficients between the K-PsyMate and the HADS subscales are shown in Table 3.

    Table 1. Item-level content validity index of the PsyMate items (n=1929).
    Item numberItemsItem-level content validity index
    1Feeling cheerful1.00
    2Feeling satisfied1.00
    3Feeling relaxed1.00
    4Feeling globally well0.83
    5Feeling lonely1.00
    6Feeling guilty1.00
    7Feeling worried1.00
    8Feeling down1.00
    9Feeling threatened1.00
    10Feeling insecure1.00
    11Feeling irritated1.00
    12Feeling frightened1.00
    13Feeling suspicious1.00
    Table 2. Factor loadings from EFAa of the K-PsyMate scale (n=1929).b
    Item numberItemsCommunalityFactor 1: Negative affectFactor 2: Positive affect
    2Feeling satisfied0.89–0.230.91
    3Feeling relaxed0.87–0.260.90
    1Feeling cheerful0.81–0.210.87
    4Feeling globally well0.80–0.320.84
    12Feeling frightened0.750.83–0.23
    10Feeling insecure0.750.82–0.27
    13Feeling suspicious0.700.82–0.19
    9Feeling threatened0.690.82–0.15
    6Feeling guilty0.630.78–0.16
    7Feeling worried0.700.77–0.33
    8Feeling down0.650.68–0.44
    11Feeling irritated0.620.62–0.48
    5Feeling lonely0.390.52–0.35

    aEFA: exploratory factor analysis.

    bValues shown in italics show factor loadings >0.50, indicating that the value fits for the given factor.

    Table 3. Correlations between the K-PsyMate scale and the HADSa (n=93).
    HADS subscales

    		K-PsyMate subscales
    Negative affect
    		Positive affect
    HADS anxiety

    		r0.465b
    		–0.348b

    		P value<.001
    		 <.001
    HADS depression

    		r0.446b
    		–0.328b

    		P value<.001
    		<.001

    aHADS: hospital anxiety and depression scale.

    bThe correlation is significant at a significance level of <.05 (2-tailed).

    Reliability

    The calculated ICC values between the first and the last measurements ranged from 0.52 to 0.80. Given the range of the 95% CI of each item, all 4 items of the positive affect factor and 6 items of the negative affect factor showed poor level of agreement. Three items of “down,” “insecure,” and “worried” in the negative affect factors were between 0.50 and 0.75 of the 95% CIs, which was considered as moderate agreement. The ICC value and the 95% CI of each item are shown in Table 4.

    Table 4. ICCa and 95% CI of the K-PsyMate scale (n=93).
    ItemsFirst measure, mean (SD)Last measure, mean (SD)ICC95% CI
    Positive affect

    		Satisfied4.37 (1.26)4.68 (1.41)0.670.497-0.779

    		Cheerful4.43 (1.32)4.76 (1.31)0.530.293-0.689

    		Globally well4.57 (1.40)4.65 (1.38)0.530.297-0.691

    		Relaxed4.57 (1.42)4.74 (1.45)0.520.277-0.682
    Negative affect

    		Down2.74 (1.61)2.43 (1.58)0.800.694-0.866

    		Insecure2.88 (1.74)2.40 (1.63)0.730.599-0.824

    		Worried3.38 (1.78)2.71 (1.67)0.670.502-0.781

    		Frightened2.55 (1.62)2.19 (1.55)0.660.490-0.776

    		Lonely3.01 (1.66)2.38 (1.62)0.650.477-0.770

    		Suspicious2.33 (1.50)2.01 (1.41)0.630.435-0.752

    		Guilty2.25 (1.46)2.04 (1.41)0.610.415-0.743

    		Threatened1.99 (1.33)1.80 (1.18)0.600.393-0.733

    		Irritated2.72 (1.70)2.26 (1.49)0.540.327-0.704

    aICC: intraclass correlation coefficient.

    Usability of the EMA Method by Using a Mobile App

    Usability was evaluated by 67 (72%) out of the 93 participants. The mean age was 42.0 (SD 10.43) years (range, 22-67 years), and 50 (74%) of the 67 participants were females. Of the 67 participants, 26 (39%) were in their thirties, whereas only 4 (6%) were in their sixties (Multimedia Appendix 1). Participants rated the accuracy of inputting their momentary mood on the app at 4.13 (SD 0.97) out of 5. Easiness in understanding questions and in operating and in inputting answers was rated at 4.12 (SD 0.88) out of 5. The enjoyment in participating in the EMA app was rated at 3.65 (SD 0.91) out of 5. The participants rated interruption by the survey alarms at 2.48 (SD 1.02) out of 5. The intention to participate or recommend others to participate in another study using EMA apps was reported as 3.25 (SD 0.95) out of 5 (Table 5).

    No difference in the usability was found in the subgroup analysis by gender, age, and marital status of the participants (Multimedia Appendix 1). Of the 67 participants, 20 (30%) wrote answers to the open-ended question in the app. Six (9%) out of 67 participants reported that they experienced discomfort owing to the small checkbox and 1 (1%) experienced network disconnection while inputting responses into the app. One participant (1%) expressed that it was a burden to answer 4 times a day.

    Table 5. Usability of the EMAa method by using a mobile app (n=67).
    Category of ratingMean (SD) of the ratingsPossible range for the ratings
    Accuracy in assessing momentary mood4.13 (0.97)1-5
    Easiness in understanding questions, operating, and inputting answers4.12 (0.88)1-5
    Enjoyment of participating via the EMA app3.65 (0.91)1-5
    Interruption by the survey alarms2.48 (1.02)1-5
    Intention to participate or recommend others to participate in another study using EMA apps3.25 (0.95)1-5

    aEMA: ecological momentary assessment.


    Principal Results

    In this study, we evaluated the psychometric properties of the K-PsyMate scale to measure moods by using the EMA method for adults with MMD since their moods are prone to change when they consciously strive to overcome negative feelings and stress, similar to patients with an unruptured cerebral aneurysm [5]. The content validity of the K-PsyMate scale evaluated by S-CVI was excellent with good I-CVI in all items determined by Polit et al [35]. The EFA results agreed with those of a previous evaluation of the PsyMate [25], which also had 2 factors: positive affect with 4 items and negative affect with 9 items. We found that the statistically significant correlations between the K-PsyMate scale and HADS were also in accordance with those reported in the previous study, which had significant correlations ranging from 0.4 to 0.7 when compared with the HADS scores [25].

    In this study, the agreement between the measurements evaluated by the ICC values of each item was poor to moderate. This poor to moderate level of agreement suggests that the measurements may have been affected by context variables since participants were requested to measure in an uncontrolled natural real-life environment [9,29]. This finding is in accordance with the results of a study on the Dutch version of the PsyMate, which presented sensitivity to the change of mood over time and applicability for patients in an ambulatory mental health setting in the Netherlands [25]. The poor ICCs may reflect the dynamics of affect fluctuating and changing over time [41] and the variability in the anxious and depressive mood of patients with MMD [7,42].

    Considering the characteristics of the EMA data, we tried a 2-level confirmatory factor analysis [43] as an alternative method, wherein latent variables of negative affect and positive affect were assessed at the individual level. However, it did not achieve convergence, and the maximum log pseudo-likelihood stayed the same at –45854.379 from the 20th iteration. Convergence rates for 2-level confirmatory factor analysis models are known to be predicted by all possible interactions between the reliability condition (poor to moderate level by ICCs in this study), number of clusters (93 individual clusters in this study), and cluster size (ranging from 3 to 28 in this study) [44]. Multilevel confirmatory factor analysis cannot be applied to this study in which individuals form groups of interest with small cluster size and with low ICCs between measurements in single-level analysis.

    The result of the usability evaluation in this study suggested that the EMA app has a good feasibility with accuracy in assessing mood and without much interruption and it may be applicable, regardless of gender or age, as no statistical difference of usability was found in the subgroup analysis.

    Limitations

    This study has the following limitations. First, we conducted the study by using a limited sample size. However, the psychometric properties of the scale could be analyzed by including 1929 measures obtained through the EMA method. Second, we recruited participants who only used the Android operating system, for which the mobile app was developed [22]. Hence, patients who used other operating systems such as the iPhone operating system were not included in this study. Nevertheless, the study is still meaningful in that we contacted about 1.2% of the target population with MMD in South Korea when considering the overall prevalence of 16.1 patients per 100,000 in 2011 [3]. Third, in this study, we included the responses of the participants who answered more than three times in the total duration of the study to evaluate the ability to capture the changes. This implies that our study results may vary according to participants who answered less than three times or did not answer at all. A qualitative study on participants’ experience with the EMA method or research on psychometric properties of the K-PsyMate scale for different populations such as patients with other disorders and symptoms will be an interesting topic for future study.

    Conclusion

    This study provides evidence that the K-PsyMate scale has good validity but poor to moderate level of agreement between the measurements. The poor to moderate level of agreement evaluated by ICCs may reflect the characteristics of the EMA data collected in real and natural living situations and environments. Our study shows that the K-PsyMate scale uploaded in a mobile app can be a valid and reliable tool for evaluating the momentary mood of patients with MMD because using a mobile app is convenient for patients as they are familiar with their own smartphones, which they use in their daily lives.

    Acknowledgments

    This study was supported by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (2017R1D1A1B03030706).

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Characteristics of the participants and subgroup analysis of the usability of the ecological momentary assessment method by using a mobile app (n=67).

    PDF File (Adobe PDF File), 385 KB
    1. Baba T, Houkin K, Kuroda S. Novel epidemiological features of moyamoya disease. J Neurol Neurosurg Psychiatry 2008 Aug;79(8):900-904. [CrossRef] [Medline]
    2. Festa JR, Schwarz LR, Pliskin N, Cullum CM, Lacritz L, Charbel FT, et al. Neurocognitive dysfunction in adult moyamoya disease. J Neurol 2010 May;257(5):806-815. [CrossRef] [Medline]
    3. Kim JS. Moyamoya Disease: Epidemiology, Clinical Features, and Diagnosis. J Stroke 2016 Jan;18(1):2-11 [FREE Full text] [CrossRef] [Medline]
    4. Koivunen R, Harno H, Tatlisumak T, Putaala J. Depression, anxiety, and cognitive functioning after intracerebral hemorrhage. Acta Neurol Scand 2015 Sep;132(3):179-184. [CrossRef] [Medline]
    5. Pala A, Pawlikowski A, Brand C, Schmitz B, Wirtz CR, König R, et al. Quality of Life After Treatment of Unruptured Intracranial Aneurysms. World Neurosurg 2019 Jan;121:e54-e59. [CrossRef] [Medline]
    6. Ahn IM, Park D, Hann HJ, Kim KH, Kim HJ, Ahn HS. Incidence, prevalence, and survival of moyamoya disease in Korea: a nationwide, population-based study. Stroke 2014 Apr;45(4):1090-1095. [CrossRef] [Medline]
    7. Liu C, Yi X, Li T, Xu L, Hu M, Zhang S, et al. Associations of depression, anxiety and PTSD with neurological disability and cognitive impairment in survivors of moyamoya disease. Psychol Health Med 2019 Jan;24(1):43-50. [CrossRef] [Medline]
    8. Ryu GW, Yang YS, Choi M, Shim KW. Lived experiences of adult patients with moyamoya disease: A qualitative case study. Jpn J Nurs Sci 2020 Mar 09:e12332. [CrossRef] [Medline]
    9. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol 2008;4:1-32. [CrossRef] [Medline]
    10. Carpenter RW, Wycoff AM, Trull TJ. Ambulatory Assessment. Assessment 2016 Aug;23(4):414-424 [FREE Full text] [CrossRef] [Medline]
    11. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess 2009 Dec;21(4):463-475. [CrossRef] [Medline]
    12. Bolger N, Laurenceau J. Intensive longitudinal methods: an introduction to diary experience sampling research. New York: Guilford Press; 2013.
    13. Cristóbal-Narváez P, Sheinbaum T, Ballespí S, Mitjavila M, Myin-Germeys I, Kwapil TR, et al. Impact of Adverse Childhood Experiences on Psychotic-Like Symptoms and Stress Reactivity in Daily Life in Nonclinical Young Adults. PLoS One 2016;11(4):e0153557. [CrossRef] [Medline]
    14. Wichers M, Simons CJP, Kramer IMA, Hartmann JA, Lothmann C, Myin-Germeys I, et al. Momentary assessment technology as a tool to help patients with depression help themselves. Acta Psychiatr Scand 2011 Oct;124(4):262-272. [CrossRef] [Medline]
    15. Dogan E, Sander C, Wagner X, Hegerl U, Kohls E. Smartphone-Based Monitoring of Objective and Subjective Data in Affective Disorders: Where Are We and Where Are We Going? Systematic Review. J Med Internet Res 2017 Jul 24;19(7):e262 [FREE Full text] [CrossRef] [Medline]
    16. Walz LC, Nauta MH, Aan Het Rot M. Experience sampling and ecological momentary assessment for studying the daily lives of patients with anxiety disorders: a systematic review. J Anxiety Disord 2014 Dec;28(8):925-937. [CrossRef] [Medline]
    17. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013 Nov 15;15(11):e247 [FREE Full text] [CrossRef] [Medline]
    18. Yang YS, Ryu GW, Choi M. Methodological Strategies for Ecological Momentary Assessment to Evaluate Mood and Stress in Adult Patients Using Mobile Phones: Systematic Review. JMIR Mhealth Uhealth 2019 Apr 01;7(4):e11215 [FREE Full text] [CrossRef] [Medline]
    19. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Feb;15(Pt 1):1-39 [FREE Full text] [CrossRef] [Medline]
    20. Freedman MJ, Lester KM, McNamara C, Milby JB, Schumacher JE. Cell phones for ecological momentary assessment with cocaine-addicted homeless patients in treatment. J Subst Abuse Treat 2006 Mar;30(2):105-111. [CrossRef] [Medline]
    21. Swendeman D, Ramanathan N, Baetscher L, Medich M, Scheffler A, Comulada WS, et al. Smartphone self-monitoring to support self-management among people living with HIV: perceived benefits and theory of change from a mixed-methods randomized pilot study. J Acquir Immune Defic Syndr 2015 May 01;69 Suppl 1:S80-S91 [FREE Full text] [CrossRef] [Medline]
    22. Yang YS, Ryu GW, Han I, Oh S, Choi M. Ecological Momentary Assessment Using Smartphone-Based Mobile Application for Affect and Stress Assessment. Healthc Inform Res 2018 Oct;24(4):381-386 [FREE Full text] [CrossRef] [Medline]
    23. Taylor K, Silver L. Smartphone ownership is growing rapidly around the world, but not always equally? In emerging economies, technology use still much more common among young people and the well-educated. Pew Research Center: Global Attitudes and Trends.: Pew Research; 2019 Feb 05.   URL: https:/​/www.​pewresearch.org/​global/​2019/​02/​05/​smartphone-ownership-is-growing-rapidly-around-the-world-but-not-always-equally/​ [accessed 2020-05-20]
    24. e-Country indicators. National government monitoring indicators. Seoul, Republic of Korea: National Indicator System   URL: http://www.index.go.kr/main.do [accessed 2020-05-21]
    25. Verhagen SJW, Berben JA, Leue C, Marsman A, Delespaul PAEG, van Os J, et al. Demonstrating the reliability of transdiagnostic mHealth Routine Outcome Monitoring in mental health services using experience sampling technology. PLoS One 2017;12(10):e0186294 [FREE Full text] [CrossRef] [Medline]
    26. Lenaert B, Colombi M, van Heugten C, Rasquin S, Kasanova Z, Ponds R. Exploring the feasibility and usability of the experience sampling method to examine the daily lives of patients with acquired brain injury. Neuropsychol Rehabil 2019 Jun;29(5):754-766. [CrossRef] [Medline]
    27. Myin-Germeys I, Birchwood M, Kwapil T. From environment to therapy in psychosis: a real-world momentary assessment approach. Schizophr Bull 2011 Mar;37(2):244-247 [FREE Full text] [CrossRef] [Medline]
    28. Kang Y, Na D, Hahn S. A validity study on the Korean mini-mental state examination (K-MMSE) in dementia patients. J Kor Neurol Assoc. 1997.   URL: http://jkna.org/upload/pdf/9702006.pdf [accessed 2020-04-05]
    29. Yang YS, Ryu GW, Park CG, Yeom I, Shim KW, Choi M. Mood and Stress Evaluation of Adult Patients With Moyamoya Disease in Korea: Ecological Momentary Assessment Method Using a Mobile Phone App. JMIR Mhealth Uhealth 2020 May 25;8(5):e17034 [FREE Full text] [CrossRef] [Medline]
    30. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983 Jun;67(6):361-370. [CrossRef] [Medline]
    31. Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res 2002 Feb;52(2):69-77. [CrossRef] [Medline]
    32. Oh S, Min K, Park D. A study on the standardization of the Hospital Anxiety and Depression Scale for Koreans - A comparison of normal, depressed and anxious groups. J Korean Neuropsychiatr Assoc. 1999.   URL: http://journal.kisep.com/pdf/002/1999/0021999026.pdf [accessed 2020-03-04]
    33. Jones KK, Zenk SN, McDonald A, Corte C. Experiences of African-American Women with Smartphone-Based Ecological Momentary Assessment. Public Health Nurs 2016 Jul;33(4):371-380. [CrossRef] [Medline]
    34. Start with the PsyMate™ yourself. PsyMate: Insight into daily life.   URL: http://www.psymate.eu [accessed 2020-04-01]
    35. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health 2006 Oct;29(5):489-497. [CrossRef] [Medline]
    36. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016 Jun;15(2):155-163 [FREE Full text] [CrossRef] [Medline]
    37. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods 1996;1(1):30-46. [CrossRef]
    38. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979 Mar;86(2):420-428. [CrossRef] [Medline]
    39. Portney LG, Watkins MP. Foundations of clinical research: applications to practice. Philadelphia, United States: F. A. Davis Company; 2015.
    40. Hoffman L. Longitudinal analysis: Modeling within-person fluctuation and change. Abingdon, United Kingdom: Routledge; 1 edition; Jan 30, 2015.
    41. Crowe E, Daly M, Delaney L, Carroll S, Malone KM. The intra-day dynamics of affect, self-esteem, tiredness, and suicidality in Major Depression. Psychiatry Res 2019 Sep;279:98-108. [CrossRef] [Medline]
    42. Yang YS, Ryu GW, Yeom I, Shim KW, Choi M. Stress and mood of adults with moyamoya disease: A cross-sectional study. Nurs Health Sci (in press) 2020 Apr 26. [CrossRef] [Medline]
    43. Geldhof GJ, Preacher KJ, Zyphur MJ. Reliability estimation in a multilevel confirmatory factor analysis framework. Psychol Methods 2014 Mar;19(1):72-91. [CrossRef] [Medline]
    44. Muthén B, Asparouhov T. Beyond multilevel regression modeling: Multilevel analysis in a general latent variable framework. Handbook of advanced multilevel analysis. New York: Taylor & Francis; 2011.   URL: https://www.statmodel.com/download/multilevelVersion2.pdf [accessed 2020-03-06]

    ‎
    CVI: content validity index
    EFA: exploratory factor analysis
    EMA: ecological momentary assessment
    HADS: hospital anxiety and depression scale
    ICC: intraclass correlation coefficient
    I-CVI: item-level content validity index
    K-PsyMate: Korean version of the PsyMate
    MMD: moyamoya disease
    S-CVI: scale-level content validity index

    Edited by G Eysenbach; submitted 23.01.20; peer-reviewed by A Krishnakumar, R Adhikary; comments to author 31.03.20; revised version received 21.05.20; accepted 27.05.20; published 21.07.20

    Copyright

    ©Yong Sook Yang, Gi Wook Ryu, Philippe A E G Delespaul, Mona Choi. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 21.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.