Test-retest Reliability and Sensitivity to Change of a New Fall Risk Assessment System: A Pilot Study
Article information
Abstract
Background
The new fall risk assessment (FRA) system is a composite and comprehensive assessment tool developed to predict the risk of falls. The aim of this pilot study was to examine the new FRA system’s test-retest reliability and sensitivity to change in community-dwelling older adults.
Methods
This was an observational study with a test-retest design and an 8-week fall prevention exercise program. A sample of 28 community-dwelling older adults with a mean age of 73.0 years (range, 65–80 years) participated in the study. The new FRA system was administered twice within a 7-day period for test-retest reliability expressed as intraclass correlation coefficient (ICC) and standard error of measurement (SEM) assessment. Eighteen subjects of them completed the 8-week fall prevention exercise intervention to evaluate the new FRA system’s sensitivity to change.
Results
In the evaluation of interrater reliability for the new FRA system, the ICC (95% confidence interval) of the total score was 0.77 (0.47–0.98), with good reliability. The SEM was 11.61 for the total FRA score. A good to excellent reliability was observed, with ICC levels of 0.73 to 0.91 for the 4 composite scores of the new FRA system. Following the 8-week exercise intervention, the mean total FRA score (effect size, 0.58) significantly increased (p=0.028).
Conclusion
The new FRA system has generally moderate to excellent interrater reliability and reliable sensitivity to change in community-dwelling older adults. Our findings provide support for the reliability of the new FRA system in healthy older adults without a fall history.
INTRODUCTION
Falls are relatively common in the elderly population, with approximately one-third of adults aged over 65 years experiencing falls at least once a year.1) One-quarter to one-half of all falls among community-dwelling older adults result in injury, 10% to 15% of falls lead to serious injuries, and 2% to 6% of falls are associated with fractures.2) The causes of falls are multifactorial and the risk of falling appears to increase with the number of risk factors. Multiple chronic illnesses such as stroke, Parkinson disease, and dementia and the use of psychoactive medications have been shown to be important predictors for falls.3) Aside from these extrinsic factors, epidemiological studies have shown that muscle weakness, especially low muscular strength, is inversely associated with the risk of falling in older adults.4–7) Lower-extremity weakness and balance impairment are the most important intrinsic factors for falls in community-dwelling older adults.8)
Currently, numerous simple and composite physical performance tests such as the short physical performance battery (SPPB), Berg balance scale (BBS), and Timed Up and Go (TUG) test as well as postural sway and reaction time have been used to screen older adults for fall risks.9–12) For comprehensive fall risk assessment, the Physiological Profile Assessment has been developed to determine the physiological fall risk based on 5 measures such as visual contrast sensitivity, lower limb proprioception, quadriceps strength, simple reaction time, and postural sway.3) A potential assessment tool in relation to falls in elderly individuals based on physical functional assessment instruments has also become common. Currently, computerized balance function assessments are increasingly used in the clinical setting in the screening of older adults to assess fall risk and examine balance dysfunction.13,14) However, they are not standardized within or across settings and lack computerized comprehensive fall risk assessment tools.
The Fall Risk Assessment (FRA) system is a new instrument recently developed by Inbody Co., Ltd. (Seoul, Korea) to predict the risk of falls.15) The balance-measuring instrument in the FRA system is used to evaluate the sensory, nervous, and musculoskeletal systems and integrated balance abilities. The FRA system is a composite and comprehensive assessment tool developed to predict the risk of falls and is composed of three machines: the balance measuring instrument, the InBody body composition analyzer (Inbody Co., Ltd.), and a leg muscle dynamometer. The FRA system multiplies each system’s standardized score by proposed weighting values to calculate the composite FRA score. The success rate in tracking moving targets well predicted a lower 6-month fall rate,15) and reaction times showed significant relationships with SPPB and BBS in community-dwelling older adults.16)
The purpose of this pilot study was to examine a new FRA system’s test-retest reliability and sensitivity to change in community-dwelling older adults.
MATERIALS AND METHODS
1. Study Participants
This was an observational study with a test-retest design and an 8-week fall prevention exercise program. Participants were recruited through advertisements at a senior center in Dongdaemun-gu, Seoul, Korea. The participants were community-dwelling older adults aged 65 years or older who were capable of communication and ambulation. The following exclusion criteria were applied: history of falls within the previous 12 months, cardiac surgery, history of myocardial infarction, and history of stroke within the previous 6 months; serious orthopedic problems such as fracture, deformity, and severe osteoarthritis; severe visual impairments or visual field defects; severe back pain; uncontrolled hypertension (systolic blood pressure >150 mmHg or diastolic blood pressure >95 mmHg); hypoglycemia (fasting plasma glucose <70 mg/dL); history of orthostatic fainting and syncope; heart disease such as myocardial infarction and heart failure; liver failure; renal failure on dialysis; and the use of psychiatric and neurological disorders such as schizophrenia, alcohol dependence, and eating disorder. A total of 31 subjects participated in this study. Twenty-eight subjects who underwent test-retest measurement of the FRA system 1 week apart were included in this study. Furthermore, the sensitivity to change was examined in 20 subjects randomly assigned to the fall prevention program group with an 8-week center-based exercise interventional trial to improve physical performance. Eighteen subjects were evaluated pre- and postintervention of the FRA system.
This study protocol was approved by the Institutional Review Board of the Clinical Research Ethics Committee of the Kyung Hee University Medical Center and all subjects provided written informed consent (approval number: 2017-07-033).
2. FRA System
The FRA system is a composite and comprehensive assessment tool developed to predict the risk of falls and composed of three machines: the balance measuring instrument, the InBody body composition analyzer, and a leg strength dynamometer. Generally, the most important physiological factors associated with falls in older adults are balance impairment and muscle weakness.17,18) The aim of the systems approach to the clinical assessment of balance, such as sensory organization and biomechanical and motor coordination constraints, is to determine the underlying cause of balance deficit.19) The FRA system for balance disorders is used to evaluate these systems using the modified clinical test of sensory interaction in balance (mCTSIB),20) reaction time test,21) and integrated balance ability test,15) whereas the linear distance from the initial point of the center of pressure (COP) is measured using the balance-measuring instrument. The FRA system for muscle weakness is used to test leg muscle strength22) and leg muscle mass.23) The FRA system takes approximately 15–20 minutes to administer for each test after a simple demonstration of how to complete each item, all with practice trials.
1) Sensory score
The mCTSIB is designed to assess how well an older adult is using different sensory inputs when one or more sensory systems are compromised. In this test, the participants were asked to maintain their balance through 4 different conditions (COP sway with eyes open on a firm surface, COP sway with eyes closed on a firm surface, COP sway with eyes open on a foam surface, and COP sway with eyes closed on a foam surface) while standing on the force platform. In the first condition (condition 1), all relevant sensory systems (i.e., visual, somatosensory, and vestibular) were available for maintaining balance. In condition 2, the participant only maintained balance by relying on the somatosensory and vestibular systems. In condition 3, the participant relied only on the visual and vestibular systems. Finally, in condition 4, the visual input was blocked and the somatosensory information was compromised; thus, the subject was forced to rely only on the vestibular system to maintain balance and an upright posture. During each condition, the FRA system measured the linear distance from the initial point of the COP to the swayed point of the COP in the specific quadrants (left forward, right forward, left back, and right back) of the board every 16 msec for a total of 30 seconds. The sway index was presented as the standard deviation displacement of the COP sway. The sensory score was calculated by weighting the sway index score of conditions 2, 3, and 4 by 70%, 10%, and 20%, respectively.
2) Integrated balance ability score
The integrated balance ability test comprised the shifting velocity test and target tracking test. The shifting velocity test measured how fast the participants shifted their COP. The target tracking test measured how successfully the participants pursued a moving target and moved their COP to that target. The integrated balance ability score was calculated as the mean of the shifting velocity score and target tracking score.
3) Reaction time score
The reaction times of the fingers and feet were also examined on the force platforms. At the beginning of the test, 4 fan-shaped buttons appeared on the upper portion of the screen and the subject’s fingers and feet were placed on the semicircular button on the lower center of the screen. The test was repeated 10 times and the average value was used to calculate the reaction time and latent reaction time scores. The reaction time score was obtained from the mean value of the reaction time scores of the fingers and feet.
4) Musculoskeletal score
The leg muscle mass was measured using the InBody body composition analyzer, which uses the bioelectrical impedance analysis. The lean mass of both legs was obtained as the leg muscle mass score. Muscle strengths as isokinetic knee extension and flexion were measured twice for each leg and twice for both legs (extension at 60° angle and flexion at 30° angle) using a leg strength dynamometer (IB-LS, InBody Co., Ltd.). The maximum of the 2 measured values was considered in the analysis. The musculoskeletal score was derived from the mean of the leg muscle mass score and the composite leg muscle strength score.
5) FRA score
The mean value was set as 70 for all the scoring systems and 15 was added or subtracted for every standard deviation increment or decrement, respectively. The composite FRA score was produced by weighting 10% to the sensory score, 20% to the reaction time score, 40% to the musculoskeletal score, and 30% to the integrated balance ability score per the company’s instructions.
3. Physical Performance Assessment
1) Handgrip strength
Grip strength was measured using a dynamometer (GRIP-D, T.K.K. 5401, Takei Scientific Instruments Co., Ltd., Nigata, Japan) with participants in a standing position with arms hanging naturally at their sides. Each hand was tested twice, alternating hands between trials with 60-second rests between measurements on the same hand. Absolute grip strength was calculated as the sum of the greatest measurement from each hand and expressed in kilograms (to the nearest 0.1 kg).
2) Short Physical Performance Battery
The SPPB consists of 3 standing balance measures (tandem, semitandem, and side-by-side stands), 5-chair-stand time, and usual gait speed. For standing balance, the participants were first asked to remain standing with their feet as close together as possible, then in a semitandem position, and finally in a tandem position. Each position had to be held for 10 seconds. The chair sit-to-stand test measured the time required to stand up and sit down 5 times as quickly as possible from a straight-backed armchair. The participants were asked to complete 5 repetitive chair stands after first demonstrating the ability to rise once from the chair with arms folded across their chests. For the usual gait speed test, the participant was asked to walk a 7-m distance twice. A walk tester beside the participant simultaneously collected the participant’s gait speed using a manual stopwatch. A stopwatch was used to measure the time taken to walk a 4-m distance from when a foot (toe) first crossed the ground after the 1.5-m line to when a foot (toe) crossed the ground after the 5.5-m line. Each test was scored from 0 to 4 based on the normative scores obtained from the Established Population for Epidemiologic Studies of the Elderly. The scores were summed to obtain a total score ranging from 0 to 12.10)
3) Timed Up and Go
Participants were asked to rise from an armchair of standard height, and a 3-m distance was marked.12) The starting position was sitting with hands resting on the arms of the chair. The participants crossed the line before turning around and walking back to sit down in the chair again. They were instructed to perform the TUG at their own comfortable and safe walking paces. The TUG was started when the participant’s back lost contact with the back of the chair and stopped when their buttocks were in contact with the seat of the chair again. Participants performed 2 trials with the results averaged to the nearest 0.01 second.
4. Test-Retest Reliability
To calculate the test-retest reliability of the FRA, these measures were performed on two separate occasions 7 days apart. The tests were carried out by 2 experienced testers and each tester had the same subjects in the test and the same re-test day to reduce interexaminer errors.
5. Sensitivity to Change
Sensitivity to change was examined in the subjects randomly assigned to the fall prevention program group with an 8-week, center-based exercise interventional trial to improve physical performance. The observations were performed pre- and postintervention of the FRA system. The fall prevention exercise intervention comprised an 8-week comprehensive program.24) The individualized exercise intensity was maintained at approximately 8–14 on the Borg Rate of Perceived Exertion Scale. The low-to-moderate-intensity program included exercises to improve balance and mobility, gait training, stretching, muscle strengthening, and recreational activities. The balance and mobility exercises consisted of activities such as standing in various postures, walking with various challenges such as obstacles, sit-to-stand movements, standing on one leg, multidirectional weight shifts, and rapid knee raises while standing. The strengthening exercises were performed in a progressive sequence from the seated to the standing positions. Each individual’s ability to increase the low-to-moderate intensity was assessed by the principal investigator and exercise instructor. The fall prevention exercise was 60 minutes and was held at the Kyung Hee University Medical Center twice per week for 2 months. The exercise session included a 5-minute stretching warm-up of the neck, shoulders, lower back, hips, knees, and ankles; 30-minute strengthening exercises; 20-minute balance and gait training; and a 5-minute cool-down.
6. Statistical Analysis
Study population characteristics were analyzed using descriptive statistics, including mean ± standard deviation (SD) and frequency. The ICC was used to assess the inter-rater reliability. The ICC (2,1 for absolute agreement) with 95% confidence intervals (CIs) was calculated for the FRA composite score and each subscore was used to analyze test-retest reliability. ICC values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability. Furthermore, the standard error of the measurement (SEM) was calculated using the following equation: SEM=SD×√(1-ICC). The observational study was designed to examine whether the FRA was sensitive to changes in older adults. To analyze the changes within the study group after the intervention, Wilcoxon signed-rank tests were used. The effect size over the intervention period for changes in FRA scores was calculated for each group using the following equation: effect size = (meanPost−meanPre)/SDPre. Effect sizes of 0.2, 0.5, and 0.8 represent small, moderate, and large effects, respectively. Statistical analyses were performed using IBM SPSS Statistics ver. 23.0 (IBM Co., Armonk, NY, USA). For all tests, the level of statistical significance was set at p<0.05.
RESULTS
A sample of 28 community-dwelling older adults (26 women and 2 men), with a mean age 73.0 ± 5.9 years (range, 65–80 years) qualified and participated in this study. The participants’ characteristics are presented in Table 1. All participants walked outdoors on a regular basis and none had reported a fall in the last year. The participants’ SPPB scores ranged between 7 and 12.
Sample characteristics (n=28)
Table 2 presents the results of the test-retest reliability analyses for the FRA. In the evaluation of the interrater reliability for the FRA, the ICC (95% CI) of the total score was 0.77 (0.47–0.98), with good reliability found for the FRA score. The SEM was 11.61 for the total FRA score. The sensory score demonstrated good interrater reliability (ICC, 0.79; 95% CI, 0.54–0.90) with an SEM of 8.55, whereas poor to moderate reliability was found for the 4 different conditions. COP sway with eyes open on a firm surface had the lowest ICC value. The integrated balance ability score had good interrater reliability (ICC, 0.78; 95% CI, 0.49–0.90), with an SEM of 3.66. The success rate in the shifting velocity score demonstrated a good interrater reliability (ICC, 0.81; 95% CI, 0.58–0.91), with an SEM of 5.08%. The reaction time score demonstrated moderate interrater reliability (ICC, 0.73; 95% CI, 0.39–0.88), with an SEM of 3.22. The reaction time of the feet (ICC, 0.74; 95% CI, 0.43–0.88) had a higher reliability than the reaction time of the fingers (ICC, 0.46; 95% CI, −0.07 to 0.74). The ICC value indicated excellent reliability for the musculoskeletal score (0.91; 95% CI, 0.76–0.96), with an SEM of 3.10. Good to excellent reliability was observed, with ICCs of 0.89 and 0.90 for the left and right leg extension strength scores, respectively. The ICCs for interrater reliability of the left and right leg flexion strength scores were 0.54 and 0.74, respectively.
Test-retest reliability of the new fall risk assessment system (n=28)
Table 3 and Fig. 1 show the sensitivity to change for the FRA of an 8-week exercise intervention. Following the exercise intervention, the mean total FRA score (effect size, 0.58) differed significantly between baseline and post-intervention assessments (p=0.028). The integrated balance ability score (effect size, 0.58) improved significantly (p=0.037).
Sensitivity to change of the new fall risk assessment system before and after the exercise intervention (n=18)
New fall risk assessment total scores pre- and postintervention.
DISCUSSION
Based on the results of our preliminary reliability study, the new FRA system demonstrated generally moderate to excellent interrater reliability in community-dwelling older adults. Moreover, the total FRA score showed sensitivity to change through the implementation of an exercise intervention. These findings suggest that the new FRA system may be a useful tool with reliability for the quantification of fall risk. In addition, its assessment instruments are able to detect change over time or the effects of a fall prevention program in clinical settings.
The sensory system identifies any increase in body sway during stance associated with altering visual or surface somatosensory information for control of standing balance.25) The CTSIB was originally developed by Shumway-Cook and Horak26) in 1986. Boughen et al.27) reported a substantial level of interrater reliability of the 4 conditions of the mCTSIB (center of gravity location) tested on the NeuroCom Balance Master (kappa range, 0.80–0.85) performed on 2 occasions separated by at least 2 weeks in 63 community-dwelling women (age range, 40–80 years). The sensory system of the new FRA system was derived from mCTSIB measured on the force platform instrument with vision. The sensory system score measured the linear distance from the initial point of COP to the sway point of COP in the 2-dimensional plane every 16 msec for a total of 30 seconds. Based on the literature, COP-based parameters have different levels of reliability.28–30) For example, Ford-Smith et al.29) assessed the test-retest reliability of the Sensory Organization Test using a force platform across 6 sensory conditions in 40 noninstitutionalized adults over the age of 65 years. The ICCs ranged from 0.15 to 0.70 during the first week-to-week trial of the Sensory Organization Test balance test. The visual stimuli conditions were altered by moving the visual screen; the subjects’ anterior/posterior sway showed poor reliability (ICC of 0.15 and 0.27) in the first trial and average scores. In 49 typically developing children,30) the SD of anterior-posterior COP data using an portable force plate in 2 test sessions on 2 different days showed poor reliability, with an ICC of 0.34 for the condition of standing with eyes open on a firm surface. Our study showed that the COP sway with eyes open had the lowest ICC value. In normal conditions, a standing subject oscillates more with eyes closed than with eyes open. A possible explanation could be that the learning effects were affected by the order of measurement on the force plate.31) The sensory system of the new FRA system was performed in the order of assessment from conditions 1 to 4 in a single test session. Further studies are needed that average the measurements from trial repetitions using a random sequence of the mCTSIB consisting of 4 conditions to increase the reliability values.29) However, the sensory system score demonstrated good interrater reliability (ICC, 0.79) with a low SEM. In this context, the sensory system of the new FRA system as the sway index score was derived from the sway indexes of conditions 2, 3, and 4, which allows clinicians to identify bias in the 3 sensory (somatosensory, visual, and vestibular) inputs involved in postural stability during a steady-state balance assessment.26) The integrated balance ability score had good interrater reliability with a low SEM. The integrated balance ability test comprised the shifting velocity and target tracking tests. The success rate in the shifting velocity of the COP to the target score demonstrated excellent interrater reliability. The target tracking test of the new FRA system measured how successfully the participants pursued a moving target and moved their COP to that target. A round target was displayed on the screen, moving at speeds of 1, 2, 3, 4, or 5 cm/sec to the left and right, front and rear, diagonal A (right front to left rear), and diagonal B (left front to right rear). Dynamic balance refers to automatic postural responses to center-of-gravity disturbances caused by movement, and a test to track a moving target with COP has been suggested as a dynamic balance test.32) Harro and Garacia13) reported similar findings in a sample of 46 healthy community-dwelling older adults with a mean age of 67.67 (5.1) years with no history of falls. The investigators reported that administering the test within a 10-day period for test-retest reliability of limits of stability (LOS) endpoint excursion measurement demonstrated good test-retest reliability (ICC, 0.77; 95% CI, 0.56–0.87), with an SEM of 9.0. The LOS test of dynamic balance assesses an individual’s weight-shifting ability and voluntary LOS to 8 directional targets set at 100% of the theoretical LOS. Our study findings support the reliability of dynamic balance in healthy older adults.
The reaction time score of the new FRA system was calculated as the mean value of the reaction time scores of the fingers and feet. The reaction time was defined as the time from the cue signal until the corresponding virtual target was hit by the hand or foot. Processing speed, the general ability of the brain to quickly and efficiently process information, declines with age and especially when tasks are more complex.33) This study showed that the reliability of the reaction time of the feet was higher than that of the reaction time of the fingers. Our study results are consistent with those of previous research. Sampaio et al.34) reported that the reaction time using a finger press as the response demonstrated poor intrarater reliability (ICC, 0.25) and moderate interrater reliability (ICC, 0.54) in a sample of 10 community-dwelling older adults with an average age of 75.2 years (range, 67–87 years). The choice stepping reaction time test has good interday test-retest reliability (ICC, 0.74; 95% CI, 0.45–0.88) in older people.35) These 2 reaction time tests examine different aspects of postural responses based on eye-hand or eye-foot response time as the underlying mechanisms. Typically, the reaction time of the fingers is faster than that of the feet.36) Based on our findings, we suggest that further studies on different reliabilities of the response time between the fingers and feet in older people should be conducted with several factors. Finally, the current study showed excellent reliability for the musculoskeletal score of the new FRA system. Symons et al.37) reported that the test-retest reliability (2 separate test days) of a single-session isokinetic and isometric strength testing protocol in healthy older men displayed good relative reliability (ICC >0.84). Our findings showed that the reliability of the knee extension strength was higher than that of the knee flexion strength. This may explain the difference in knee extensors and flexors reported by Hartmann et al.38). Movements toward gravity such as isokinetic bilateral knee flexion are more difficult to perform than movements against gravity. This study showed that the right knee flexion strength has a higher reliability than the left knee flexion strength. Bergamin et al.39) observed the lowest correlation on the left ankle isokinetic dorsal flexion test (ICC, 0.36) and left ankle isokinetic plantar flexion test (ICC, 0.60). This may be explained in terms of dominance; in fact, the vast majority of the participants was right-handed and the right leg was dominant.
For use in clinical settings, it is essential that FRA instruments detect changes over time or the effects of intervention studies. This study showed significant improvements with a moderate effect size in the mean total FRA score and integrated balance ability score in an 8-week center-based exercise interventional trial to improve physical performance. These results indicate that the FRA instruments are able to detect change over time or the effects of a fall prevention program in clinical settings.
This study had several limitations. First, our use of the new FRA system among community-dwelling volunteer older adults included many women in a primary care setting, which may not be applicable to populations in nursing homes, hospitals, and other institutions, which limits the generalizability of our findings. Moreover, although the study was powered to determine an acceptable level of agreement, the sample size was small; thus, the reliability coefficients may not be robust. Second, our findings were specific to healthy older adults without a fall history. It was not the purpose of this study to examine whether the new FRA system sensitively detected fall risk in older adults with a fall history. Further research is needed to assess the intra- and interrater reliability of the new FRA system in the examination of intrinsic balance deficits in elderly individuals with a fall history. Finally, this observation study, which examined sensitivity to change, did not include a control group. Further randomized controlled trials are needed to confirm the capacity of the new FRA system in individuals with and without a fall history.
In conclusion, the findings of this study support the reliability of the new FRA system in healthy older adults without a fall history. The new FRA system has generally moderate to excellent interrater reliability in community-dwelling older adults. In addition, the total FRA score showed sensitivity to change through the implementation of an exercise intervention. Although the new FRA system provides diagnostic information on fall risk that may be informative in older adults who are identified as high fall risk, it may also be a useful tool to identify physical functional status decline and recovery in clinical and community settings.
ACKNOWLEDGMENTS
This study was supported by a grant from InBody Co., Ltd. (Seoul, Korea) and the Seoul Business Agency, a government-affiliated organization (PS150024).
Notes
The researchers claim no conflicts of interest.