3054 — A Method of Preparing the Physical Component Scale of the SF-12 for IRT Analysis

Kudel I (Cincinnati VAMC & University of Cincinnati), Edwards MC (Ohio State University), Tsevat J (Cincinnati VAMC & University of Cincinnati), Veterans Aging Cohort Study Team (VA Connecticut Healthcare System )

Objectives:
Item response theory (IRT) has become the gold standard for assessing the psychometric properties of self-reported outcome measures because its mathematical properties allow one to conduct analyses not possible using other techniques, such as identifying and correcting noninvariance and developing crosswalks between measures. However, one assumption of IRT local independence among items in a measure is often not met. The purpose of this analysis is to demonstrate a method to rectify this violation without sacrificing any items from a measure.

Methods:
We analyzed the 6-item Physical Component Summary (PCS) scale of the SF-12 in 2938 veterans with HIV who participated in the Veterans’ Aging Cohort Study. We applied a 4-step method: 1) performing confirmatory factor analysis (CFA) to determine unidimensionality, 2) combining items violating local independence into “testlets,” 3) performing IRT using a graded response model (GRM) and a nominal response model (NRM), and 4) analyzing those items and testlets by using GRM. Data were analyzed by using Mplus and Multilog software.

Results:
Step 1) CFA found good fit only after each 2-item scale (Physical Function and Role Physical) was covaried, indicating that each grouping violated local independence. Step 2) Two pairs of testlets were created, with the two possible responses for each of the Role Physical items combined to derive four categories (no/no, yes/no, no/yes, yes/yes). The Physical Function items had three response choices each for a total of nine groupings. The order of the middle responses of the testlets wasn’t clear. Step 3) GRM modeled the two items and NRM ordered the response choices for each testlet and identified pairs of item responses that could be collapsed, reducing the nine Physical Function testlet groupings into four without a decrement in model fit (delta chi-square = 5.4,df = 4,p = 0.14). Step 4) GRM analyzed the newly ordered testlets and the two other PCS items, resulting in a scale with good marginal reliability (0.78), an IRT analogue of coefficient alpha.

Implications:
This 4-step method allows one to model the PCS of SF-12 with minimal loss of information.

Impacts:
The method can be extended to any measure used to assess a self-reported outcome.