JMU JMIR Mhealth Uhealth JMIR mHealth and uHealth 2291-5222 JMIR Publications Toronto, Canada v8i7e17926 32706707 10.2196/17926 Original Paper Original Paper Psychometric Properties of the Korean Version of the PsyMate Scale Using a Smartphone App: Ecological Momentary Assessment Study Eysenbach Gunther Krishnakumar Arjun Adhikary Rishi Yang Yong Sook MSN 1 https://orcid.org/0000-0003-3372-071X Ryu Gi Wook MPH 1 https://orcid.org/0000-0002-4533-7788 Delespaul Philippe A E G PhD 2 https://orcid.org/0000-0001-7420-0898 Choi Mona PhD 1
Mo-Im Kim Nursing Research Institute Yonsei University College of Nursing 50 Yonsei-ro, Seodaemun-gu Seoul, 03722 Republic of Korea 82 2 2228 3341 monachoi@yuhs.ac
https://orcid.org/0000-0003-4694-0359 Mo-Im Kim Nursing Research Institute Yonsei University College of Nursing Seoul Republic of Korea Department of Psychiatry and Neuropsychology, Faculty of Health Medicine and Lifesciences Maastricht University Maastricht Netherlands Corresponding Author: Mona Choi monachoi@yuhs.ac 7 2020 21 7 2020 8 7 e17926 23 1 2020 31 3 2020 21 5 2020 27 5 2020 ©Yong Sook Yang, Gi Wook Ryu, Philippe A E G Delespaul, Mona Choi. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 21.07.2020. 2020

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.

 Background

Ecological momentary assessment (EMA) is a method for capturing the changes in the variables in daily life with increased accuracy and decreased recall bias. The PsyMate scale assesses momentary moods in daily life and can be used in various settings.

 Objective

The aim of this study was to develop a Korean version of the PsyMate (K-PsyMate) scale and evaluate its psychometric properties by using the EMA method in patients with moyamoya disease (MMD) in South Korea.

 Methods

Patients with MMD aged over 18 years were recruited from July 2018 to January 2019 at the inpatient and outpatient departments of a university hospital in South Korea. The K-PsyMate scale comprising 13 items was developed following a translation/back translation approach of the English version and loaded onto a mobile app. Participants were instructed to enter their moods 4 times a day for 7 consecutive days. Content validity index, factor analysis, and Pearson’s correlation were performed for validity analysis. For reliability, intraclass correlation coefficients between the first and last measurements were estimated by mean rating, absolute agreement, and a 2-way mixed-effects model. Usability was analyzed through a descriptive analysis, 2-tailed t test, and analysis of variance, and the results were confirmed by Mann-Whitney U test and the Kruskal-Wallis test, as the dependent variable was not normally distributed.

 Results

In total, 1929 assessments from 93 patients were analyzed. The mean age of the participants was 40.59 (SD 10.06) years, and 66 (71%) of the 93 participants were women. Content validity was excellent as content validity index was 0.99, and 2 factors, negative affect and positive affect, were derived by an exploratory factor analysis. The correlations between the subdomains of the K-PsyMate scale and Hospital Anxiety and Depression Scale were significant (P<.001). The agreement between the first and last measurements was poor to moderate according to the obtained intraclass correlation coefficient values. Usability was evaluated by 67 (72%) out of the 93 participants. The participants rated the accuracy of assessing their momentary moods on the app at 4.13 (SD 0.97), easiness in understanding questions, operating, and inputting answers at 4.12 (SD 0.88), and interruption by the survey alarms at 2.48 (SD 1.02) out of 5.

 Conclusions

The K-PsyMate scale has good validity but poor to moderate agreement, which reflects the characteristics of the EMA data collected in real and natural living environments without control. The findings of our study show that the K-PsyMate scale uploaded in a mobile app can be a valid and reliable tool for evaluating the momentary mood of patients with MMD because using a mobile app is convenient and patients are familiar with their own smartphones, which they use in their daily lives.

 psychometrics reliability validity PsyMate scale ecological momentary assessment Introduction

Moyamoya disease (MMD), also called as spontaneous occlusion of the circle of Willis, is a chronic progressive cerebrovascular disease with no known cause and is characterized by fine collateral vessel networks that resemble hazy cloud-like puffs of smoke in the brain [1,2]. MMD occurs worldwide with high prevalence in East Asian countries such as Japan and Korea [3]. Unlike MMD in children, MMD in adults is associated with potential rupture or ischemia of the blood vessels in the brain, similar to that reported in other cerebrovascular diseases [4,5], and this can lead to high mortality [6]. The clinical features of patients with MMD often appear as mental deterioration, headache, and speech impairment [6], which are affected by mood changes. Adults with MMD also show high levels of anxiety and depression [7] and their moods change over time during the course of life [8]. Negative mood and mood changes in adults with MMD may affect their motivation for proper health behaviors and eventually affect the health outcomes of patients with similar cerebrovascular diseases [7]. Therefore, monitoring the moods of adults with MMD can be critical for both patients and health care personnel.

Measuring moods may help people better understand their emotional states. The measuring tool to self-assess moods should be understandable and comfortable to use; at the same time, the tool should be able to assess the mood accurately and capture the fluctuations and changes in the mood reliably since mood varies according to context and time [9,10]. To avoid mood changes caused by the tool itself, the participants should be allowed to respond to the questionnaire in a flexible and friendly atmosphere of their choice.

Traditionally, mood has been assessed by retrospective measures, including pen and paper tests [11], but these methods lead to recall bias and risk of reduced accuracy [9,10]. However, momentary assessments such as ecological momentary assessment (EMA) or experience sampling methods allow increased accuracy and reduced bias because these assessments are performed in real-life environments and through repeated assessments at multiple times [9-11]. The EMA method is known for its effectiveness in capturing changes [12] and understanding daily changes in psychological states such as mood and stress [13,14]. It is broadly used to assess the psychological aspects of the participants with [15-17] and without mental health problems [18]. The increased prevalence of smartphones and advanced mobile technology has made people more familiar with the use of smartphones in their daily lives, and EMA methods using smartphones are available [19-22]. The prevalence of smartphone use in Korean adults is about 95% [23], and about 96.4% of the population under the age of 70 years in Korea can access internet services [24]. In a study conducted with healthy participants in Korea, the EMA method was evaluated as comfortable to use and easy to understand when the mood was assessed using a mobile app installed in their own smartphones [22].

Various measurement tools using mobile technology have been developed to measure moods in daily life. The PsyMate scale was developed to assess momentary moods relating to the daily life of the participants [25], and it has been used in various settings and populations [26,27]. In this study, we aimed to develop a Korean version of the PsyMate (K-PsyMate) scale, upload it on a mobile app to measure the momentary moods of Korean adults with MMD, and examine the psychometric properties of this scale.

 Methods Participants

Patients with MMD diagnosed via magnetic resonance angiography and aged over 18 years were recruited from July 2018 to January 2019 at the outpatient department or the inpatient wards of a university hospital in Seoul, South Korea. Patients who could not communicate owing to neurological problems were excluded by a physician and the physician’s assistant who were involved in the treatment of patients with MMD. The Korean version of the Mini-Mental State Examination [28] was used to assess the cognitive impairment of the participants after acquiring the permission to use it from the provider. Participants with a score under 24 in the Korean version of the Mini-Mental State Examination were excluded. Participants who used operating systems other than the Android system were also excluded, since the developed mobile app was available only for the Android operating system as described in the previous study [22]. Informed consent was obtained from all the participants when they enrolled in the study. This study was approved by the Institutional Review Board of the Yonsei University Health System (No. 4-2018-0385).

 Measurements Participants’ Characteristics and Baseline Mood

All participants were asked to provide their demographic and disease-specific information at the baseline measurement through a questionnaire that was developed by our research team. Demographics included age, gender, marital status, people living with, job status, monthly household income, education level, and smoking and drinking history. Detailed information on the participants’ demographics has been presented in our previous work [29]. Regarding the baseline mood, the Hospital Anxiety and Depression Scale (HADS) [30] was administered after obtaining permission to use it. The HADS is known as a globally reliable and valid scale for assessing mental health in hospital settings [31]. It consists of 14 items, 7 for anxiety and 7 for depression, and it is measured on a 4-point scale ranging from 0 to 3. A higher value indicates a higher level of anxiety and depression. In this study, we administered the Korean version of HADS [32], for which Cronbach α was .89 and .86 for anxiety and depression subscales, respectively [32]. The HADS was assessed to compare the momentary negative affect and positive affect that were measured by the K-PsyMate scale, as described in a previous study [25].

 K-PsyMate Scale for Measuring Momentary Mood

The mood scale of the K-PsyMate was used for the EMA section of this study. This scale consists of 13 items, of which 4 items measure positive affect and 9 items assess negative affect. The positive affect comprised feeling “cheerful,” “satisfied,” “relaxed,” and “globally well,” and the negative affect comprised feeling “lonely,” “guilty,” “worried,” “down,” “threatened,” “insecure,” “irritated,” “frightened,” and “suspicious.” These items were rated on a 7-point scale, with “1” indicating “not at all” and “7” indicating “very much.” Momentary mood was measured while considering the environmental context, taking into account what the participants were doing, where they were, and with whom they were when responding to the questionnaire.

 Usability Measure

The usability of the EMA method on the mobile phone was evaluated by the participants at the end of the survey. A 10-item questionnaire was uploaded in the app: 2 items for accuracy, 3 items for easiness, 2 items for enjoyment, 2 items for intention, and 1 item for interruption. It was adapted from a previous study [33] with the permission from the developer and scored on a 5-point scale (1=strongly disagree to 5=strongly agree). An open-ended question was included for the participants to offer any additional comment or opinion. The Cronbach α of the scale in this study was .74.

 Procedure Translation of the PsyMate

We freely downloaded the English version of the PsyMate scale [34] with permission from the copyright holder. To develop the K-PsyMate scale, we used the translation/back translation approach. We employed a committee approach for translation, in which we consulted 3 nursing professors who were fluent in both English and Korean to translate the English version of PsyMate [25]. Through this method, we identified language and cultural differences in selecting words. The translated PsyMate items were prepared for back translation during which we consulted 3 different nursing professors who were fluent in both English and Korean. The research team compared the back-translated PsyMate with the original English version to confirm that the translated Korean version was accurate. After confirming the translated Korean version through back translation, the research team finalized the K-PsyMate draft. This draft was then evaluated by a linguist of the Korean language prior to determining the final version. Eventually, the K-PsyMate version was finalized by the research team.

 Data Collection

While obtaining informed consent from each patient, members of the research team obtained baseline data on demographic and disease-specific information by using the structured questionnaire and on mood status by using the HADS. They helped each participant install the mobile app on their mobile phone and register themselves, and they provided time for practice. Participants selected the EMA sampling period to assess momentary moods according to their convenience. They were guided by an alarm to respond 4 times a day for 7 consecutive days (4 times × 7 days = 28 times/person) at 60-minute blocked semirandom intervals. Notifications were sent in the morning between 8 AM and 9 AM, early afternoon between noon and 1 pm, evening between 5 PM and 6 PM, and at night between 9 PM and 10 PM. A reminder for each scheduled measurement was sent to the participants who did not input the response within 45 minutes after the first notification. The contact numbers and the email addresses of the members of the research team were given to the participants to contact the team if they had any questions or if they faced any difficulties during the assessment period. Finally, patients were requested to complete a 10-item survey at the end of the EMA session on the usability of the EMA method using a mobile app, including an open-ended question.

 Statistical Analysis

Data were analyzed using the SPSS v.25.0 (IBM Corp). We used descriptive statistics to analyze the baseline data of the participants’ characteristics, the HADS scores, and the positive affect and negative affect of the K-PsyMate scale measured by the EMA method. We applied a 2-tailed t test and analysis of variance to examine the differences between groups according to the general characteristics. Regarding validity tests, we used the content validity index (CVI), which calculated scale-level CVI (S-CVI) and each item-level CVI (I-CVI). To assess CVI, each item was appraised by an expert panel consisting of 6 nursing professors. The CVI for each item was scored using a 4-point scale, where 1=not relevant, 2=somewhat relevant, 3=quite relevant, and 4=highly relevant. S-CVI and I-CVI were evaluated with the cut-off of S-CVI level at 0.90 and I-CVI at 0.78 [35]. We performed exploratory factor analysis (EFA) to examine construct validity using principal component analysis and varimax rotation. We also used Pearson’s correlation (r) to assess the association between the HADS and PsyMate in the previous study [25]. We calculated intraclass correlation coefficients (ICCs) for evaluating the test-retest reliability of the measurement tool. Reliability reflects both the degree of correlation and agreement between the measurements, and ICCs are an index that show the degree of correlation and agreement between the measurements [36]. Among the 10 different forms of ICC [37,38], we calculated the ICCs of each item between the first and the last measurement by mean rating, absolute agreement, and a 2-way mixed-effects model, since we tested absolute agreement of the mean of multiple measurements from a nonrandomly selected population [36]. The obtained ICC values of each item were interpreted by its 95% CI ranges [36]. The ICC estimates in the 95% CI were evaluated by considering ICC<0.50, 0.50≤ICC<0.75, 0.75≤ICC<0.90, and ICC≥0.90 as poor, moderate, good, and excellent reliability, respectively [39]. For usability evaluation, we performed descriptive analysis, 2-tailed t test, and analysis of variance, and we confirmed the results by Mann-Whitney U test and the Kruskal-Wallis test, as the dependent variable was not normally distributed.

 Results Participants

We recruited patients with MMD from the inpatient and outpatient departments of a tertiary hospital in South Korea. We approached 119 patients of which 109 (91.6%) agreed to participate, and they were enrolled using the mobile app. Among the 109 patients, 98 (89.9%) participated in the EMA session. The data of 93 participants who provided more than 3 measurements considered as EMA data [40] and who provided 1929 responses (74.1%) out of the 2604 scheduled assessments were analyzed. The mean age of the participants was 40.59 (SD 10.06) years, and 66 of the 93 participants (71%) were women. The HADS anxiety and HADS depression scores were 7.17 (SD 3.38) and 7.14 (SD 3.51), respectively. The overall mean negative affect and positive affect scores were 2.15 (SD 1.12) and 4.70 (SD 1.31) out of 7, respectively.

 Validity of the K-PsyMate Scale

We calculated S-CVI and I-CVI as evaluated by the expert panel. The S-CVI was 0.99 (I-CVI range 0.83-1.00), which represented an excellent level of content validity. We also calculated the S-CVIs of 2 factors of negative affect and positive affect according to the original study [25], and they were 1.00 and 0.96, respectively. The I-CVIs are presented in Table 1.

EFA was performed based on 1929 completed assessments from 93 participants. We confirmed the presence of latent factors in the observed items with excellent Kaiser-Meyer-Olkin of 0.93 and with significant Bartlett’s sphericity measures (P<.05). Two factors were derived with the eigenvalue greater than 1 with a cumulative percentage of explained variance of 71.1%. All items were assigned to either the negative affect (Factor 1) or positive affect (Factor 2) factor. These 2 factors were negatively related (r=–0.67). The results of the EFA are shown in Table 2.

The subdomains of negative affect and positive affect of the K-PsyMate scale and the HADS anxiety and depression were significantly correlated with coefficients (r) ranging between 0.33 and 0.68. The Cronbach α of the HADS anxiety and depression in this study were .84 and .79, respectively. The correlation coefficients between the K-PsyMate and the HADS subscales are shown in Table 3.

Item-level content validity index of the PsyMate items (n=1929).

Item number Items Item-level content validity index
1 Feeling cheerful 1.00
2 Feeling satisfied 1.00
3 Feeling relaxed 1.00
4 Feeling globally well 0.83
5 Feeling lonely 1.00
6 Feeling guilty 1.00
7 Feeling worried 1.00
8 Feeling down 1.00
9 Feeling threatened 1.00
10 Feeling insecure 1.00
11 Feeling irritated 1.00
12 Feeling frightened 1.00
13 Feeling suspicious 1.00

Factor loadings from EFAa of the K-PsyMate scale (n=1929).b

Item number Items Communality Factor 1: Negative affect Factor 2: Positive affect
2 Feeling satisfied 0.89 –0.23 0.91
3 Feeling relaxed 0.87 –0.26 0.90
1 Feeling cheerful 0.81 –0.21 0.87
4 Feeling globally well 0.80 –0.32 0.84
12 Feeling frightened 0.75 0.83 –0.23
10 Feeling insecure 0.75 0.82 –0.27
13 Feeling suspicious 0.70 0.82 –0.19
9 Feeling threatened 0.69 0.82 –0.15
6 Feeling guilty 0.63 0.78 –0.16
7 Feeling worried 0.70 0.77 –0.33
8 Feeling down 0.65 0.68 –0.44
11 Feeling irritated 0.62 0.62 –0.48
5 Feeling lonely 0.39 0.52 –0.35

aEFA: exploratory factor analysis.

bValues shown in italics show factor loadings >0.50, indicating that the value fits for the given factor.

Correlations between the K-PsyMate scale and the HADSa (n=93).

HADS subscales K-PsyMate subscales
Negative affect Positive affect
HADS anxiety
r 0.465b –0.348b
P value <.001 <.001
HADS depression
r 0.446b –0.328b
P value <.001 <.001

aHADS: hospital anxiety and depression scale.

bThe correlation is significant at a significance level of <.05 (2-tailed).

 Reliability

The calculated ICC values between the first and the last measurements ranged from 0.52 to 0.80. Given the range of the 95% CI of each item, all 4 items of the positive affect factor and 6 items of the negative affect factor showed poor level of agreement. Three items of “down,” “insecure,” and “worried” in the negative affect factors were between 0.50 and 0.75 of the 95% CIs, which was considered as moderate agreement. The ICC value and the 95% CI of each item are shown in Table 4.

ICCa and 95% CI of the K-PsyMate scale (n=93).

Items First measure, mean (SD) Last measure, mean (SD) ICC 95% CI
Positive affect
Satisfied 4.37 (1.26) 4.68 (1.41) 0.67 0.497-0.779
Cheerful 4.43 (1.32) 4.76 (1.31) 0.53 0.293-0.689
Globally well 4.57 (1.40) 4.65 (1.38) 0.53 0.297-0.691
Relaxed 4.57 (1.42) 4.74 (1.45) 0.52 0.277-0.682
Negative affect
Down 2.74 (1.61) 2.43 (1.58) 0.80 0.694-0.866
Insecure 2.88 (1.74) 2.40 (1.63) 0.73 0.599-0.824
Worried 3.38 (1.78) 2.71 (1.67) 0.67 0.502-0.781
Frightened 2.55 (1.62) 2.19 (1.55) 0.66 0.490-0.776
Lonely 3.01 (1.66) 2.38 (1.62) 0.65 0.477-0.770
Suspicious 2.33 (1.50) 2.01 (1.41) 0.63 0.435-0.752
Guilty 2.25 (1.46) 2.04 (1.41) 0.61 0.415-0.743
Threatened 1.99 (1.33) 1.80 (1.18) 0.60 0.393-0.733
Irritated 2.72 (1.70) 2.26 (1.49) 0.54 0.327-0.704

aICC: intraclass correlation coefficient.

 Usability of the EMA Method by Using a Mobile App

Usability was evaluated by 67 (72%) out of the 93 participants. The mean age was 42.0 (SD 10.43) years (range, 22-67 years), and 50 (74%) of the 67 participants were females. Of the 67 participants, 26 (39%) were in their thirties, whereas only 4 (6%) were in their sixties (Multimedia Appendix 1). Participants rated the accuracy of inputting their momentary mood on the app at 4.13 (SD 0.97) out of 5. Easiness in understanding questions and in operating and in inputting answers was rated at 4.12 (SD 0.88) out of 5. The enjoyment in participating in the EMA app was rated at 3.65 (SD 0.91) out of 5. The participants rated interruption by the survey alarms at 2.48 (SD 1.02) out of 5. The intention to participate or recommend others to participate in another study using EMA apps was reported as 3.25 (SD 0.95) out of 5 (Table 5).

No difference in the usability was found in the subgroup analysis by gender, age, and marital status of the participants (Multimedia Appendix 1). Of the 67 participants, 20 (30%) wrote answers to the open-ended question in the app. Six (9%) out of 67 participants reported that they experienced discomfort owing to the small checkbox and 1 (1%) experienced network disconnection while inputting responses into the app. One participant (1%) expressed that it was a burden to answer 4 times a day.

Usability of the EMAa method by using a mobile app (n=67).

Category of rating Mean (SD) of the ratings Possible range for the ratings
Accuracy in assessing momentary mood 4.13 (0.97) 1-5
Easiness in understanding questions, operating, and inputting answers 4.12 (0.88) 1-5
Enjoyment of participating via the EMA app 3.65 (0.91) 1-5
Interruption by the survey alarms 2.48 (1.02) 1-5
Intention to participate or recommend others to participate in another study using EMA apps 3.25 (0.95) 1-5

aEMA: ecological momentary assessment.

 Discussion Principal Results

In this study, we evaluated the psychometric properties of the K-PsyMate scale to measure moods by using the EMA method for adults with MMD since their moods are prone to change when they consciously strive to overcome negative feelings and stress, similar to patients with an unruptured cerebral aneurysm [5]. The content validity of the K-PsyMate scale evaluated by S-CVI was excellent with good I-CVI in all items determined by Polit et al [35]. The EFA results agreed with those of a previous evaluation of the PsyMate [25], which also had 2 factors: positive affect with 4 items and negative affect with 9 items. We found that the statistically significant correlations between the K-PsyMate scale and HADS were also in accordance with those reported in the previous study, which had significant correlations ranging from 0.4 to 0.7 when compared with the HADS scores [25].

In this study, the agreement between the measurements evaluated by the ICC values of each item was poor to moderate. This poor to moderate level of agreement suggests that the measurements may have been affected by context variables since participants were requested to measure in an uncontrolled natural real-life environment [9,29]. This finding is in accordance with the results of a study on the Dutch version of the PsyMate, which presented sensitivity to the change of mood over time and applicability for patients in an ambulatory mental health setting in the Netherlands [25]. The poor ICCs may reflect the dynamics of affect fluctuating and changing over time [41] and the variability in the anxious and depressive mood of patients with MMD [7,42].

Considering the characteristics of the EMA data, we tried a 2-level confirmatory factor analysis [43] as an alternative method, wherein latent variables of negative affect and positive affect were assessed at the individual level. However, it did not achieve convergence, and the maximum log pseudo-likelihood stayed the same at –45854.379 from the 20th iteration. Convergence rates for 2-level confirmatory factor analysis models are known to be predicted by all possible interactions between the reliability condition (poor to moderate level by ICCs in this study), number of clusters (93 individual clusters in this study), and cluster size (ranging from 3 to 28 in this study) [44]. Multilevel confirmatory factor analysis cannot be applied to this study in which individuals form groups of interest with small cluster size and with low ICCs between measurements in single-level analysis.

The result of the usability evaluation in this study suggested that the EMA app has a good feasibility with accuracy in assessing mood and without much interruption and it may be applicable, regardless of gender or age, as no statistical difference of usability was found in the subgroup analysis.

 Limitations

This study has the following limitations. First, we conducted the study by using a limited sample size. However, the psychometric properties of the scale could be analyzed by including 1929 measures obtained through the EMA method. Second, we recruited participants who only used the Android operating system, for which the mobile app was developed [22]. Hence, patients who used other operating systems such as the iPhone operating system were not included in this study. Nevertheless, the study is still meaningful in that we contacted about 1.2% of the target population with MMD in South Korea when considering the overall prevalence of 16.1 patients per 100,000 in 2011 [3]. Third, in this study, we included the responses of the participants who answered more than three times in the total duration of the study to evaluate the ability to capture the changes. This implies that our study results may vary according to participants who answered less than three times or did not answer at all. A qualitative study on participants’ experience with the EMA method or research on psychometric properties of the K-PsyMate scale for different populations such as patients with other disorders and symptoms will be an interesting topic for future study.

 Conclusion

This study provides evidence that the K-PsyMate scale has good validity but poor to moderate level of agreement between the measurements. The poor to moderate level of agreement evaluated by ICCs may reflect the characteristics of the EMA data collected in real and natural living situations and environments. Our study shows that the K-PsyMate scale uploaded in a mobile app can be a valid and reliable tool for evaluating the momentary mood of patients with MMD because using a mobile app is convenient for patients as they are familiar with their own smartphones, which they use in their daily lives.

Characteristics of the participants and subgroup analysis of the usability of the ecological momentary assessment method by using a mobile app (n=67).

 Abbreviations CVI

content validity index

 EFA

exploratory factor analysis

 EMA

ecological momentary assessment

 HADS

hospital anxiety and depression scale

 ICC

intraclass correlation coefficient

 I-CVI

item-level content validity index

 K-PsyMate

Korean version of the PsyMate

 MMD

moyamoya disease

 S-CVI

scale-level content validity index

This study was supported by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (2017R1D1A1B03030706).

None declared.

 Baba T Houkin K Kuroda S Novel epidemiological features of moyamoya disease J Neurol Neurosurg Psychiatry 2008 08 79 8 900 4 10.1136/jnnp.2007.130666 18077479 jnnp.2007.130666 Festa JR Schwarz LR Pliskin N Cullum CM Lacritz L Charbel FT Mathews D Starke RM Connolly ES Marshall RS Lazar RM Neurocognitive dysfunction in adult moyamoya disease J Neurol 2010 05 257 5 806 15 10.1007/s00415-009-5424-8 20033200 Kim JS Moyamoya Disease: Epidemiology, Clinical Features, and Diagnosis J Stroke 2016 01 18 1 2 11 10.5853/jos.2015.01627 26846755 jos-2015-01627 PMC4747069 Koivunen R Harno H Tatlisumak T Putaala J Depression, anxiety, and cognitive functioning after intracerebral hemorrhage Acta Neurol Scand 2015 09 132 3 179 84 10.1111/ane.12367 25639837 Pala A Pawlikowski A Brand C Schmitz B Wirtz CR König Ralph Kapapa T Quality of Life After Treatment of Unruptured Intracranial Aneurysms World Neurosurg 2019 01 121 e54 e59 10.1016/j.wneu.2018.09.010 30244183 S1878-8750(18)32030-8 Ahn IM Park D Hann HJ Kim KH Kim HJ Ahn HS Incidence, prevalence, and survival of moyamoya disease in Korea: a nationwide, population-based study Stroke 2014 04 45 4 1090 5 10.1161/STROKEAHA.113.004273 24595588 STROKEAHA.113.004273 Liu C Yi X Li T Xu L Hu M Zhang S Jiang R Associations of depression, anxiety and PTSD with neurological disability and cognitive impairment in survivors of moyamoya disease Psychol Health Med 2019 01 24 1 43 50 10.1080/13548506.2018.1467024 29678126 Ryu GW Yang YS Choi M Shim KW Lived experiences of adult patients with moyamoya disease: A qualitative case study Jpn J Nurs Sci 2020 03 09 e12332 10.1111/jjns.12332 32153137 Shiffman S Stone AA Hufford MR Ecological momentary assessment Annu Rev Clin Psychol 2008 4 1 32 10.1146/annurev.clinpsy.3.022806.091415 18509902 Carpenter RW Wycoff AM Trull TJ Ambulatory Assessment Assessment 2016 08 23 4 414 424 10.1177/1073191116632341 26887808 1073191116632341 PMC6410721 Ebner-Priemer UW Trull TJ Ecological momentary assessment of mood disorders and mood dysregulation Psychol Assess 2009 12 21 4 463 75 10.1037/a0017075 19947781 2009-22537-002 Bolger N Laurenceau J Intensive longitudinal methods: an introduction to diary experience sampling research 2013 New York Guilford Press Cristóbal-Narváez Paula Sheinbaum T Ballespí Sergi Mitjavila M Myin-Germeys I Kwapil TR Barrantes-Vidal N Impact of Adverse Childhood Experiences on Psychotic-Like Symptoms and Stress Reactivity in Daily Life in Nonclinical Young Adults PLoS One 2016 11 4 e0153557 10.1371/journal.pone.0153557 27082442 PONE-D-15-53809 PMC4833319 Wichers M Simons CJP Kramer IMA Hartmann JA Lothmann C Myin-Germeys I van Bemmel A L Peeters F Delespaul P van Os J Momentary assessment technology as a tool to help patients with depression help themselves Acta Psychiatr Scand 2011 10 124 4 262 72 10.1111/j.1600-0447.2011.01749.x 21838742 Dogan E Sander C Wagner X Hegerl U Kohls E Smartphone-Based Monitoring of Objective and Subjective Data in Affective Disorders: Where Are We and Where Are We Going? Systematic Review J Med Internet Res 2017 07 24 19 7 e262 10.2196/jmir.7006 28739561 v19i7e262 PMC5547249 Walz LC Nauta MH Aan Het Rot Marije Experience sampling and ecological momentary assessment for studying the daily lives of patients with anxiety disorders: a systematic review J Anxiety Disord 2014 12 28 8 925 37 10.1016/j.janxdis.2014.09.022 25445083 S0887-6185(14)00146-7 Donker T Petrie K Proudfoot J Clarke J Birch M Christensen H Smartphones for smarter delivery of mental health programs: a systematic review J Med Internet Res 2013 11 15 15 11 e247 10.2196/jmir.2791 24240579 v15i11e247 PMC3841358 Yang YS Ryu GW Choi M Methodological Strategies for Ecological Momentary Assessment to Evaluate Mood and Stress in Adult Patients Using Mobile Phones: Systematic Review JMIR Mhealth Uhealth 2019 04 01 7 4 e11215 10.2196/11215 30932866 v7i4e11215 PMC6462888 Heron KE Smyth JM Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments Br J Health Psychol 2010 02 15 Pt 1 1 39 10.1348/135910709X466063 19646331 bjhp696 PMC2800172 Freedman MJ Lester KM McNamara C Milby JB Schumacher JE Cell phones for ecological momentary assessment with cocaine-addicted homeless patients in treatment J Subst Abuse Treat 2006 03 30 2 105 11 10.1016/j.jsat.2005.10.005 16490673 S0740-5472(05)00203-5 Swendeman D Ramanathan N Baetscher L Medich M Scheffler A Comulada WS Estrin D Smartphone self-monitoring to support self-management among people living with HIV: perceived benefits and theory of change from a mixed-methods randomized pilot study J Acquir Immune Defic Syndr 2015 05 01 69 Suppl 1 S80 91 10.1097/QAI.0000000000000570 25867783 00126334-201505011-00011 PMC4485442 Yang YS Ryu GW Han I Oh S Choi M Ecological Momentary Assessment Using Smartphone-Based Mobile Application for Affect and Stress Assessment Healthc Inform Res 2018 10 24 4 381 386 10.4258/hir.2018.24.4.381 30443427 PMC6230530 Taylor K Silver L Smartphone ownership is growing rapidly around the world, but not always equally? In emerging economies, technology use still much more common among young people and the well-educated Pew Research Center: Global Attitudes and Trends 2019 02 05 2020-05-20 Pew Research https://www.pewresearch.org/global/2019/02/05/smartphone-ownership-is-growing-rapidly-around-the-world-but-not-always-equally/ e-Country indicators National government monitoring indicators 2020-05-21 Seoul, Republic of Korea National Indicator System http://www.index.go.kr/main.do Verhagen SJW Berben JA Leue C Marsman A Delespaul PAEG van Os J Lousberg R Demonstrating the reliability of transdiagnostic mHealth Routine Outcome Monitoring in mental health services using experience sampling technology PLoS One 2017 12 10 e0186294 10.1371/journal.pone.0186294 29023568 PONE-D-17-10926 PMC5638497 Lenaert B Colombi M van Heugten C Rasquin S Kasanova Z Ponds R Exploring the feasibility and usability of the experience sampling method to examine the daily lives of patients with acquired brain injury Neuropsychol Rehabil 2019 06 29 5 754 766 10.1080/09602011.2017.1330214 28562164 Myin-Germeys I Birchwood M Kwapil T From environment to therapy in psychosis: a real-world momentary assessment approach Schizophr Bull 2011 03 37 2 244 7 10.1093/schbul/sbq164 21224492 sbq164 PMC3044621 Kang Y Na D Hahn S A validity study on the Korean mini-mental state examination (K-MMSE) in dementia patients J Kor Neurol Assoc 1997 2020-04-05 http://jkna.org/upload/pdf/9702006.pdf Yang YS Ryu GW Park CG Yeom I Shim KW Choi M Mood and Stress Evaluation of Adult Patients With Moyamoya Disease in Korea: Ecological Momentary Assessment Method Using a Mobile Phone App JMIR Mhealth Uhealth 2020 05 25 8 5 e17034 10.2196/17034 32449687 v8i5e17034 PMC7281123 Zigmond AS Snaith RP The hospital anxiety and depression scale Acta Psychiatr Scand 1983 06 67 6 361 70 10.1111/j.1600-0447.1983.tb09716.x 6880820 Bjelland I Dahl AA Haug TT Neckelmann D The validity of the Hospital Anxiety and Depression Scale. An updated literature review J Psychosom Res 2002 02 52 2 69 77 10.1016/s0022-3999(01)00296-3 11832252 S0022399901002963 Oh S Min K Park D A study on the standardization of the Hospital Anxiety and Depression Scale for Koreans - A comparison of normal, depressed and anxious groups J Korean Neuropsychiatr Assoc 1999 2020-03-04 http://journal.kisep.com/pdf/002/1999/0021999026.pdf Jones KK Zenk SN McDonald A Corte C Experiences of African-American Women with Smartphone-Based Ecological Momentary Assessment Public Health Nurs 2016 07 33 4 371 80 10.1111/phn.12239 26530781 Start with the PsyMate™ yourself PsyMate: Insight into daily life 2020-04-01 http://www.psymate.eu Polit DF Beck CT The content validity index: are you sure you know what's being reported? Critique and recommendations Res Nurs Health 2006 10 29 5 489 97 10.1002/nur.20147 16977646 Koo TK Li MY A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research J Chiropr Med 2016 06 15 2 155 63 10.1016/j.jcm.2016.02.012 27330520 S1556-3707(16)00015-8 PMC4913118 McGraw KO Wong SP Forming inferences about some intraclass correlation coefficients Psychological Methods 1996 1 1 30 46 10.1037/1082-989X.1.1.30 Shrout PE Fleiss JL Intraclass correlations: uses in assessing rater reliability Psychol Bull 1979 03 86 2 420 8 10.1037//0033-2909.86.2.420 18839484 Portney LG Watkins MP Foundations of clinical research: applications to practice 2015 Philadelphia, United States F. A. Davis Company Hoffman L Longitudinal analysis: Modeling within-person fluctuation and change 2015 01 30 Abingdon, United Kingdom Routledge; 1 edition Crowe E Daly M Delaney L Carroll S Malone KM The intra-day dynamics of affect, self-esteem, tiredness, and suicidality in Major Depression Psychiatry Res 2019 09 279 98 108 10.1016/j.psychres.2018.02.032 29661498 S0165-1781(17)30094-X Yang YS Ryu GW Yeom I Shim KW Choi M Stress and mood of adults with moyamoya disease: A cross-sectional study Nurs Health Sci (in press) 2020 04 26 10.1111/nhs.12729 32336006 Geldhof GJ Preacher KJ Zyphur MJ Reliability estimation in a multilevel confirmatory factor analysis framework Psychol Methods 2014 03 19 1 72 91 10.1037/a0032138 23646988 2013-15544-001 Muthén B Asparouhov T Beyond multilevel regression modeling: Multilevel analysis in a general latent variable framework Handbook of advanced multilevel analysis 2011 2020-03-06 New York Taylor & Francis https://www.statmodel.com/download/multilevelVersion2.pdf