BACKGROUND/RATIONALE:
Several national and multinational studies have demonstrated the importance of using failure-to-rescue (FTR) as a quality measure for acute care hospitals. FTR has been more closely linked to hospital characteristics than mortality. The use of FTR within VHA has been investigated using AHRQ PSI software (Agency for Healthcare Research and Quality, Patient Safety Indicators) which was adopted from the Silber model (reference Med Care. 1992; 30:615-629), where FTR was found to occur in 155 out of 1000 eligible deaths and was the most frequently occurring patient safety indicator (PSI). Research has also linked FTR with specific nursing characteristics such as staffing, staffing mix, and educational level. Data from VHA were often excluded from many of these studies, so relationships with VHA are relatively untested. The use of administrative data as a diagnostic tool for FTR was largely untested, thus laying the foundation for this study.

OBJECTIVE(S):
The primary objectives of the project were to test the following specific aims:

Aim 1: To evaluate the diagnostic performance (sensitivity, specificity) of ICD-9 CM codes for determining cases of failure-to-rescue among deaths of veterans.

Hypothesis: ICD-9 CM codes for identifying cases of FTR will have 60% or less sensitivity and specificity.

Aim 2: To explore the relationship between a set of candidate predictor variables and FTR as determined by chart review (gold standard).

Hypothesis: Rates of FTR as defined by the gold standard chart review will differ for different levels of the candidate predictor variables. Specifically, age, race/ethnicity, gender, secondary diagnosis ICD-9 CM codes, hospital site, unit where the death occurred, length of stay on the unit where the death occurred, total length of stay (LOS), and number of in-hospital patient unit transfers during LOS will be predictors of FTR.

METHODS:
All inpatient deaths within 30 days of admission during FY 2008 across the five VISN-7 hospitals were reviewed for eligibility in the study. Additional variables collected included demographic variables, all secondary diagnosis ICD-9 CM codes, hospital site, unit where death occurred, length of stay (LOS) in unit where death occurred, number of patient transfers within the hospital during the entire LOS, and total LOS. A chart audit was performed by trained chart abstractors to identify FTR based on information documented in the chart. Four possible outcomes were reviewed to confirm that each case: 1) met both FTR criteria by chart review and ICD-9 CM codes; 2) did not meet FTR criteria by chart review and ICD-9 CM codes; 3) met FTR criteria by chart review but not ICD-9 CM codes; or 4) did not meet FTR criteria by chart review but did by ICD-9 CM code.

For Aim 1, accuracy of the Silber model criteria was tested via sensitivity, specificity, positive and negative predictive values. In addition, a logit model of the association of individual Silber criteria with FTR was estimated.

For Aim 2, three newly created predictor models were tested relative to FTR using sensitivity and specificity analysis. Also, logit models were estimated to test the association of the newly created models with FTR.

FINDINGS/RESULTS:
The sample included 2000 patients from the 8 VAMCs in VISN 7 who died in the project period, within 30 days of admission. Of the Silber model criteria, respiratory compromise was the best performing criterion, with a sensitivity of 67.9% and specificity of 65.7%. In a logit model which included all Silber model inclusion and exclusion criteria, those statistically significant (p<0.05) were pulmonary compromise, respiratory complications and internal organ damage. Of the three newly created composite models, the best model was found to be the combination of pulmonary compromise, respiratory complications, and internal organ damage. This composite variable had the highest sensitivity (74.1%) and specificity (64.5%) as a predictor, of all three models estimated. Also, based on the logit model, this composite variable was 5 times more likely to predict FTR, having a much higher likelihood than the odds ratios from the other two models. These findings have significant implications for clinical planning.

IMPACT:
Failure-to-rescue (FTR) represents an important indicator of patient safety and quality for Veterans' health care. Accurate and efficient predictors of FTR are essential to be used in the practice setting, and as a guide for quality of nursing care for delivery to Veterans. Since findings demonstrate that ICD-9 CM codes alone do not accurately reflect adverse events, then effective quality of care interventions for avoidable deaths require alternative measures of FTR be explored, or proposals made for adjustments. By identifying the underlying problems and deploying targeted, systematic changes, the Veterans Health Administration can begin to handle unexpected deaths quickly and plan proactive preventive interventions.

The evidence in this study demonstrates that administrative data may not be a reliable reflection of clinical practice. Therefore, ICD-9 CM codes must be linked with clinical conditions as reflected in the medical record. Although preliminary ground work has been laid, more extensive study, testing laboratory and chart review data in addition to diagnostic criteria will be conducted. Additional work must address inter-rater reliability as well as the bias issues that exist when abstractors review their own hospital.

---

External Links for this Project

---

PUBLICATIONS:

None at this time.

DRA: Health Systems
DRE: Diagnosis, Prevention
Keywords: Nursing, Patient outcomes, Quality Measure
MeSH Terms: none