Session number: 1126
Abstract title: Establishing Benchmarks for Blind Rehabilitation by Calibrating Clinical and Self-Report Measures
Author(s):
JL Babcock - Southern AZ VA Medical Center
DN Head - Southern AZ VA Medical Center
PE McKnight - University of Arizona, Department of Psychology
RW Massof - Lions Vision Research and Rehabilitation Center, Wilmer Ophthalmological Institute, Johns Hopkins University
Objectives: The primary objective of this project is to establish benchmarks for VA Blind Rehabilitation (BR) inpatient training programs by calibrating two outcome instruments used to measure functional ability in geriatric patients who are blind or severely visually impaired. Calibration of the two measures permits easy transformations of values in order to establish a “gold standard” or benchmark. These benchmarks can be used to compare outcomes across the ten VA BR Centers and eventually to outpatient vision programs. The instruments calibrated are the Blind Rehabilitation Service Follow-up Outcome Survey (BRSFOutSur), a self-report measure, and the Functional Assessment of Self-reliance on Tasks (FAST), a clinical measure.
Methods: The FAST data were collected at admission and discharge for veterans who attended Southwestern Blind Rehabilitation Center. The BRSFOutSur data were collected by phone survey 3-months post-discharge for the same cohort. The databases were merged yielding approximately 300 matching cases for secondary analyses. Rasch analyses were conducted to estimate the difficulty of the assessment items and to develop a common measurement metric. Subsequent to the Rasch analysis, a generalizability study was used to identify sources of variation and under what circumstances responses are meaningful. Following these procedures, both Rasch score matching as well as classical raw score bivariate regression analyses were performed to compute and algebraic formula for converting scores from the BRSFOutSur to the FAST and vice versa.
Results: The Rasch analyses provided information about the stability of items both within a single assessment period and over two separate assessment periods (i.e., pre- and post). From these results, only those items that performed well were kept for the calibration procedures. An algebraic conversion algorithm was determined and the implications and their use are discussed in detail. The analyses also identified the personal characteristics that moderate change and item characteristics. From these findings, a patient profiling system was developed and used to establish defensible benchmarks.
Conclusions: The calibration methods identified interchangeable items to serve as benchmarks. This approach to benchmarking provides a new method for measuring program performance and effectiveness.
Impact statement: Calibrating items provides a common measurement metric and a foundation for understanding measurement theory in VA Blind Rehabilitation.