This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
The International Classification of Functioning, Disability, and Health (ICF) is a unified system of functioning terminology that has been used to develop electronic health records and assessment instruments. Its application has been limited, however, by its complex terminology, numerous categories, uncertain operationalization, and the training required to use it well. Together is a mobile health app designed to extend medical support to the families of spinal cord injury (SCI) patients in China. The app’s core framework is a set of only 31 ICF categories. The app also provides rating guidelines and automatically transforms routine assessment results to the terms of the ICF qualifiers.
The goal of the research is to examine the suitability of the ICF set used in the app Together for use as an instrument for assessing the functioning of SCI patients.
A cross-sectional study was conducted including 112 SCI patients recruited before discharge from four rehabilitation centers in China between May 2018 and October 2019. Nurses used the app to assess patient functioning in face-to-face interviews. The resulting data were then subjected to Rasch analysis.
After deleting two categories (family relationships and socializing) and one personal factor (knowledge about spinal cord injury) that did not fit the Rasch model, the body functions and body structures, activities and participation, and contextual factors components of the ICF exhibited adequate fit to the Rasch model. All three demonstrated acceptable person separation indices. The 28 categories retained in the set were free of differential item functioning by gender, age, education level, or etiology.
Together overcomes some of the obstacles to practical application of the ICF. The app is a reliable assessment tool for assessing functioning after spinal cord injury.
The International Classification of Functioning, Disability, and Health (ICF) is a unified system of terminology for multidisciplinary use issued in 2001 by the World Health Organization (WHO). It provides a consensus framework for defining functioning and disability and their interrelationships with health conditions and contextual factors [
However, challenges have limited practical application of the ICF, including its relatively complex terminology and category numbering. Each ICF category has its own distinct definition, which doesn’t always accord with the prevailing medical terminology. Professionals need to be trained before using the system [
A mobile health app is a health-related software program installed on smart mobile devices that provides health information and tracks a user’s health behavior. It might also allow remote consultation [
The study team previously developed an ICF-based app called Together for the transitional care of spinal cord injury (SCI) patients in China [
Together’s core framework is a set of ICF categories that reflect the levels of functioning typical of SCI patients and help organize online assessment, standardize health guidance, and coordinate interdisciplinary collaboration. The app uses fewer and more specific categories than the normal ICF core set, focusing on the transitional care of SCI patients. Preliminary studies identified 31 ICF categories as the most useful outcome indicators in the transitional care of SCI patients, covering the major physiological, psychological, and social participation problems of SCI patients at home [
This study was part of a research program designed to document the effects of an app-based transitional care model for SCI patients at home. Rasch analysis was applied to examine the suitability of the app’s set of categories. The overall aim was to determine to what extent Together can solve problems related to using the ICF in clinical practice.
The assumption of the Rasch model is that a person with greater ability is more likely than a person with less ability to pass in relation to an item, and that an easy item is more likely to be passed than a difficult one [
A cross-sectional design was employed involving four research centers in Guangzhou, Chengdu, and Shiyan in China. The study was approved by Sun Yat-sen University’s ethics committee (file 2017ZSLYEC-0620).
The participants were recruited between May 2018 and October 2019 prior to discharge from the four research centers. The inclusion criteria were age 18 years or older, SCI according to the International Standards for Neurological Classification of Spinal Cord Injury [
The Together app was designed to help medical staff provide remote follow-up for home-dwelling SCI patients. Hospital-based nurses, physicians, and therapists responsible for the transitional care for SCI patients at home are the target users. The core functions of the app comprise online assessment, providing standard health guidance, interdisciplinary referral within the team, interaction among health staff and patients, and management of online follow-up. With the help of the app, health care personnel can assess patient performance in terms of ICF categories and remotely provide health education to patients according to the assessment results. The app can be used to refer patients to different professionals on the health care team. Weekly reminders make managing follow-up by medical staff easier.
The app’s development has been reported previously [
The 31 ICF categories address physiological functioning, psychological functioning, complications, daily living activities, social participation, adaptation to environmental factors, and personal factors. For each category, guidelines were established for converting routine clinical assessment results to the ICF qualifiers, and a standardized guidance program was formulated by the expert panel based on the knowledge-attitude-practice theory. Together is an Android app developed using the Java language. Most of the app’s functions—online assessment, providing standard health guidance, interdisciplinary referral—apply to all of the categories.
The app’s utility depends heavily on to what extent its assessment results reflect functioning differences among patients with different capabilities. This study used Rasch analysis to test the app’s suitability as an assessment instrument.
Development process of the Together app.
Medical staff communicate with the patients face to face or by telephone, assess them on each ICF category, and record the results in the app, which offers standard verbal prompts to the clinician to unify different clinicians’ assessments with respect to each ICF category. For example, the verbal prompt of the sensation of pain (b280) category was “If 0 is not painful and 10 is the most painful, how serious is your pain?” Four transformation guidelines were developed to transform the initial clinical assessment results: 0 = no problem, 1 = mild, 2 = moderate, 3 = severe, or 4 = complete problem (
Guideline 1 transforms patient information in the form of percentages to the ICF qualifiers. Using muscle power functions (b730) as an example, the 0 qualifier would indicate that all of the key muscles below the injured neurological level had power grade >3, and the 4 qualifier would indicate that 95% to 100% of key muscles had power grade <3. Guideline 2 transforms the wording of patient reports to the qualifiers. Using mobility of joint functions (b710) as an example, no limitation of joint mobility, slight limitation, moderate limitation, severe limitation, and total immobility would be rated as 0, 1, 2, 3, and 4, respectively. Guideline 3 transforms the frequency with which a problem was observed to the qualifiers. Using increased blood pressure (b4200) as an example, stable blood pressure over the past month would be rated as 0, whereas high blood pressure almost every day would receive a 4. Guideline 4 transforms assessment results generated using routine clinical instruments or standards, such as the 0-10 numerical rating scale (NRS) for pain, to the qualifiers. NRS scores of 0, 1-2, 3-4, 5-9, and 10 would be rated as 0, 1, 2, 3, and 4, respectively (
Guidelines for transforming routine assessment results to the International Classification of Functioning, Disability, and Health qualifiers.
ICFa qualifier | Guideline 1b | Guideline 2c | Guideline 3d | Guideline 4e |
0 (no problem) | 0%-4% | No, none, absent, negligible... | The person has no such problem. | NRSf for pain, MASg, NPIAPh stage, FIMi, WHOQoL-BREFj, SF-36k |
1 (mild problem) | 5%-24% | Mild, slight, low... | The problem rarely happened in the last month (<25% of the time). | NRSf for pain, MASg, NPIAPh stage, FIMi, WHOQoL-BREFj, SF-36k |
2 (moderate problem) | 25%-49% | Moderate, medium, fair... | The problem happened occasionally in the last month (<50% of the time). | NRSf for pain, MASg, NPIAPh stage, FIMi, WHOQoL-BREFj, SF-36k |
3 (severe problem) | 50%-95% | Severe, high, extreme... | The problem happened frequently in the last month (>50% of the time). | NRSf for pain, MASg, NPIAPh stage, FIMi, WHOQoL-BREFj, SF-36k |
4 (complete problem) | 96%-100% | Complete, total... | The problem happened almost every day in the last month (>95% of the time). | NRSf for pain, MASg, NPIAPh stage, FIMi, WHOQoL-BREFj, SF-36k |
aICF: International Classification of Functioning, Disability, and Health.
bGuideline 1: transforms patient information in the form of percentages to the ICF qualifiers.
cGuideline 2: transforms wording from patient reports to the ICF qualifiers.
dGuideline 3: transforms the frequency with which a problem was observed during the previous month to the ICF qualifiers.
eGuideline 4: transforms the scores of a routine clinical instrument or standards to the ICF qualifiers.
fNRS: numeric rating scale.
gMAS: Modified Ashworth Scale.
hNPIAP: National Pressure Injury Advisory Panel.
iFIM: Functional Independence Measure.
jWHOQoL-BREF: World Health Organization Quality of Life Assessment–Abbreviated.
kSF-36: 36-Item Short Form Health Survey.
The questionnaire consisted of two parts, including demographic and disease-related data such as name, gender, age, education level, diagnosis, etiology, American Spinal Injury Association Impairment Scale grade, SCI level, and duration of the disability. The information was collected by nurses in face-to-face interviews with the patients and by reviewing their medical records.
The 31 categories came from the three components body functions and body structures (15 categories), activities and participation (10 categories), and contextual factors (6 categories). Although the ICF does not classify personal factors in the contextual factors component, four personal factor items related to the psychology of SCI patients were included based on preliminary testing (acceptance of life in a wheelchair/in bed, knowledge about spinal cord injury, coping with everyday life, and adjustment to new body image) [
All nurses who participated in the study first received half a day of training involving a lecture and workshop, including an introduction to the study and how to use the app. Eligible patients were invited to participate in the study before discharge. After signing the informed consent form, demographic and disease-related data were collected by the trained nurses in face-to-face interviews and by reviewing the patients’ medical records. The nurses then assessed the patients’ performance with respect to each ICF category using the app. They did this face to face referring to the standard verbal prompts for each category in the app. The app system allows submission only after all categories have been evaluated; otherwise, the system indicates to the user that the evaluation is incomplete.
SPSS Statistics software version 21.0 (IBM Corporation) was used to analyze the demographic and disease data. RUMM2030 software (RUMM Laboratory Pty) was used to perform the Rasch analysis. For each component of the ICF set, the overall fit to a Rasch model was examined. If the overall fit was not good, poorly fitting categories were identified and deleted. Another round of Rasch analysis was then run until adequate overall fit was attained. The following properties of the ICF set were examined.
A nonsignificant value in a χ2 test for item-trait interaction, a mean within ±2.5 (SD <1.5) for the fit residuals of the items and persons indicate good overall fit to the Rasch model [
The good fit of a single category was represented by a nonsignificant χ2 test and a mean of the fit residual values within ±2.5 [
An acceptable person separation index indicates good internal consistency for the instrument and reflects the ability of the instrument to discriminate between people with different abilities. It has a range of 0 to 1, with higher values indicating a better ability (>0.7 indicates good) [
For an ideal Rasch model, no factors should influence a person’s performance regarding an item except the Rasch factors [
The demographic and disease characteristics of all 112 spinal cord injury patients are shown in
Characteristics of the study sample (n=112).
Characteristic | Value, n (%) | |
|
||
|
Male | 93 (83.0) |
|
Female | 19 (17.0) |
|
||
|
18-29 | 27 (24.1) |
|
30-39 | 19 (17.0) |
|
40-49 | 36 (32.1) |
|
50-59 | 20 (17.9) |
|
60-65 | 20 (17.9) |
|
||
|
Primary school and below | 29 (25.9) |
|
Junior high school | 47 (42.0) |
|
Senior high school | 21 (18.8) |
|
College and above | 15 (13.4) |
|
||
|
Trauma | 100 (89.3) |
|
Nontrauma | 12 (10.7) |
|
||
|
1-6 | 59 (52.7) |
|
7-12 | 40 (35.7) |
|
13-18 | 12 (10.7) |
|
19-22 | 1 (0.9) |
|
||
|
Complete injury | 57 (50.9) |
|
Incomplete injury | 55 (49.1) |
|
||
|
Cervical | 31 (27.7) |
|
Thoracic | 56 (50.0) |
|
Lumbar sacral | 25 (22.3) |
The 31 ICF categories belonged to body functions and body structures (15), activities and participation (10), and contextual factors (6). To attain adequate fit to the Rasch model for each component, categories that did not fit were deleted as multiple rounds of Rasch analysis were conducted.
Summary of results of the Rasch analyses (n=112).
Analysis and action | Item fit residual, mean (SD) | Person fit residual, mean (SD) | Overall model fita | Person separation index | |||
|
|
|
χ2 |
|
|||
|
|||||||
|
1 | Original categories | –0.41 (0.86) | –0.32 (0.62) | 41.5 | .08 | 0.50 |
|
|||||||
|
2 | Original categories | –0.48 (2.27) | –0.38 (0.80) | 75.7 | <.001b | 0.89 |
|
3 | Deleted family relationships (d760) | –0.18 (2.37) | –0.30 (0.79) | 43.3 | <.001b | 0.89 |
|
6 | Deleted family relationships (d760) and socializing (d9205) | –0.06 (1.59) | –0.23 (0.73) | 24.7 | .08 | 0.89 |
|
|||||||
|
7 | Original categories | 0.25 (1.63) | –0.45 (1.30) | 32.8 | .001b | 0.65 |
|
8 | Deleted knowledge about spinal cord injury | –0.33 (1.32) | –0.43 (1.22) | 13.6 | .19 | 0.68 |
aOverall model fit was tested using a χ2 test with a Bonferroni-adjusted
bSignificant according to the Bonferroni-adjusted
In the first-round Rasch analysis, the body functions and body structures component consisting of 15 categories exhibited a nonsignificant χ2 test result for the item-trait interaction (χ230=41.5,
Regarding the initial activities and participation component with 10 categories, in the first-round Rasch analysis, the χ2 test for the item-trait interaction yielded a significant result (χ220=75.7,
For the contextual factors component, the first-round Rasch analysis starting with 6 categories indicated poor model fit according to the χ2 test result for the item-trait interaction (χ212=32.8,
International Classification of Functioning, Disability, and Health categories retained after multiple rounds of Rasch analysis.
ICFa category | Location | Fit residual | χ2b | Transformation | |||
|
|||||||
|
1 | Sleep functions (b134) | –1.450 | –0.762 | 1.0 | .61 | 4 |
|
2 | Emotional functions (b152) | –1.281 | –1.414 | 7.9 | .02 | 3 |
|
3 | Sensation of pain (b280) | 0.308 | 1.047 | 2.4 | .30 | 4 |
|
4 | Blood vessel functions (b415) | 2.034 | 0.395 | 3.7 | .16 | 2 |
|
5 | Increased blood pressure (b4200) | 1.932 | 0.789 | 3.6 | .16 | 3 |
|
6 | Decreased blood pressure (b4201) | 0.757 | –1.106 | 1.6 | .46 | 3 |
|
7 | Immunological system functions (b435) | 1.625 | –0.922 | 1.8 | .40 | 2 |
|
8 | Respiration functions (b440) | 1.767 | –0.156 | 0.8 | .68 | 3 |
|
9 | Weight maintenance functions (b530) | 0.729 | 0.523 | 2.4 | .30 | 1 |
|
10 | Sexual functions (b640) | –2.588 | –2.086 | 5.5 | .07 | 2 |
|
11 | Procreation functions (b660) | –1.640 | –0.862 | 1.6 | .44 | 2 |
|
12 | Mobility of joint functions (b710) | –0.895 | –0.791 | 1.7 | .43 | 2 |
|
13 | Muscle power functions (b730) | –2.610 | –0.216 | 3.5 | .17 | 1 |
|
14 | Muscle tone functions (b735) | –1.026 | –0.586 | 2.7 | .25 | 4 |
|
15 | Structure of areas of skin (s810) | 2.339 | 0.046 | 1.2 | .54 | 4 |
|
|||||||
|
16 | Changing basic body position (d410) | 0.252 | –1.543 | 5.6 | .06 | 4 |
|
17 | Moving around using equipment (d465) | –0.614 | –0.274 | 2.2 | .33 | 4 |
|
18 | Washing oneself (d510) | –1.697 | –1.594 | 5.2 | .08 | 4 |
|
19 | Caring for body parts (d520) | 1.147 | –0.637 | 0.8 | .66 | 4 |
|
20 | Regulating urination (d5300) | –0.866 | 3.353 | 8.5 | .01 | 4 |
|
21 | Regulating defecation (d5301) | –1.841 | 0.713 | 0.4 | .80 | 4 |
|
22 | Dressing (d540) | 0.127 | 0.26 | 0.7 | .72 | 4 |
|
23 | Eating (d550) and drinking (d560) | 3.493 | –0.741 | 1.3 | .52 | 4 |
|
|||||||
|
24 | Assistive products and technology for personal indoor and outdoor mobility and transportation (e1201) | 1.805 | 0.585 | 2.4 | .30 | 2 |
|
25 | Design, construction, and building products and technology of buildings for private use (e155) | –0.177 | 1.793 | 2.7 | .26 | 2 |
|
26 | Acceptance of life in a wheelchair/in bed | –0.237 | –1.264 | 3.3 | .19 | 2 |
|
27 | Coping with everyday life | –1.127 | 1.305 | 1.9 | .39 | 2 |
|
28 | Adjustment to new body image | –0.263 | –0.787 | 3.3 | .19 | 2 |
aICF: International Classification of Functioning, Disability, and Health.
bGoodness of fit of each category was tested using a
cFamily relationships (d760) and socializing (d9205) were deleted because of poor fit.
dKnowledge about spinal cord injury was deleted because of poor fit.
The results of the Rasch analysis showed good fit to the Rasch model for the different components of the ICF set as implemented in the app after modification. Both overall and single-item fit were satisfactory. There was no differential item functioning for any of the ICF categories by gender, age, education level, or etiology. These results indicate the suitability of the app-based ICF set as an assessment tool for assessing the functioning of SCI patients.
The app-based ICF set is one of many forms of ICF-based electronic health records. Several previous studies have confirmed the role of ICF-based electronic health records in reflecting patient functioning and facilitating rehabilitation [
Together’s verbal prompts standardize assessment and give more consistent assessment results. The transformation guidelines operationalize the 5 ICF qualifiers simply and effectively. With the help of the app, the ICF qualifiers can automatically be matched to the initial clinical assessment results. No additional training on ICF terminology or qualifiers is needed. The process reduces the differences among assessors and makes presentation of the ICF data more convenient and intelligent.
Family relationships (d760), socializing (d9205), and knowledge about spinal cord injury were deleted. In a previous study, d760 and d9205 also exhibited poor model fit [
It is worth noting that, in contrast to the person separation index of the activities and participation component, the person separation index of the other two components was not ideal (0.5 and 0.68), which reflects the poor internal consistency of the two components. This may be related to the different measuring guidelines used. Four guidelines were used in the study to transform the input of the medical staff, based on patient information in the form of percentages, wording from patient reports, frequency with which a problem was observed, and scores on routine clinical instruments or standards. A previous study [
In interpreting these results, it is important to keep in mind that the relatively small sample may have influenced the representativeness of the results. Also, although the app was designed to assess SCI patients at home during transitional care via its remote follow-up function, the study data were collected in face-to-face interviews before the participants were discharged. Further validation with larger samples and remote assessment via the app’s communication function are needed.
This study has confirmed the suitability of the Together app as an assessment tool. With relatively fewer but more specific ICF categories, it overcomes some of the limitations related to applying the ICF and makes assessment results more reliable and consistent. The app opens up a new way to use the ICF with SCI and electronic health records. In the future, the app could be used to capture information about the functioning of SCI patients at home remotely. Such assessment results would help to monitor patients’ functional changes and differences, learn their needs, identify their problems, and provide evidences for further interventions if necessary.
Together tutorial.
International Classification of Functioning, Disability, and Health
numerical rating scale
spinal cord injury
World Health Organization
The authors would like to express special thanks to Bing Xie and Haishan Pan, the engineers who provided technical support for the development of the app. This study was supported by China’s National Natural Science Foundation (grant number 71603293) and the Fundamental Research Funds for the Central Universities (grant number 20ykpy88). The sponsors had no role in the study’s design; in the collection, analysis or interpretation of data; in writing of the report; or in the decision to submit the article for publication.
MJ drafted the manuscript and assisted with managing the app. JT was responsible for conducting the study and revising the manuscript. SX, XH, and YW helped collect the data. TL was responsible for the management of the app. TY participated in the study’s design. KL was responsible for the project design, implementation, quality control, app management, and manuscript revision. All the authors reviewed the submitted manuscript.
None declared.