3127 — Does mode of survey administration matter? Using measurement invariance to validate the mailed version of the Bereaved Family Survey (BFS)

Smith D, PROMISE Center, PVMAC; Kuzla N, PROMISE Center, PVMAC; Thorpe J, PROMISE Center, PVMAC; Scott L, PROMISE Center, PVMAC; Ersek M, PROMISE Center, PVMAC;

Objectives:
The VHA is committed to delivering high quality, Veteran-centered care. As one way to assess achievement of this goal, the VHA evaluates nationally outcomes of end-of-life (EOL) care via the validated Bereaved Family Survey (BFS). Initially the BFS was administered by telephone, but transitioned to a mail survey in September 2012. To make valid comparisons over time, the BFS should measure identical constructs across modes of administration. When true, this is called measurement invariance (MI). The objective of this study was to validate the mail version and the test for MI across the two administration modes.

Methods:
Telephone and mailed versions of the BFS were validated separately between October 2009 and September 2013. MI was evaluated using a series of confirmatory factor analyses (CFA). Additional psychometric evaluation included Cronbach's alpha, assessing floor/ceiling effects, and evaluating criterion validity by examining associations between individual BFS items and the BFS Performance Measure (BFS-PM). Construct validity was evaluated by examining differences between BFS scores for groups with and without quality care indicators (e.g., receipt of a palliative care consult).

Results:
Our sample consisted of BFS data from 35,682 decedents (27,109 telephone surveys; 8,573 mail surveys). BFS-PM scores for both modes of administration were slightly skewed, with a predominance of higher scores (telephone = 58%; mail = 63%). A series of CFA models demonstrated dimensional, configural, metric, and factor mean invariance across administration modes. The BFS also had good homogeneity for both administration methods (telephone: Cronbach's alpha = 0.81; mail: Cronbach's alpha = 0.83). Significant associations between the BFS-PM and each BFS item were found for both modes of administration (P < 0.001 for all). BFS item scores were higher when a patient received EOL quality care indicators regardless of mode of administration.

Implications:
These findings demonstrate the measurement invariance and robust psychometric properties of the BFS across administration modes.

Impacts:
Establishing measurement invariance and validity of the mail administration method is critical for continued use of the BFS in accurately assessing, and ultimately, improving quality of care.