BACKGROUND/RATIONALE:
Valid and reliable metrics are essential for the evaluation of any implementation project. Though several implementation models propose such metrics, eHealth implementation has several unique characteristics that warrant a separate investigation of the most effective metrics.

OBJECTIVE(S):
The goal of this proposal was to conduct a thorough review of potential metrics that could be used in any study using patient facing eHealth interventions and create a standardized array of recommended metrics that will support both VA operations and eHealth studies.

Specific aims:

1. To identify and evaluate existing metrics (instruments and measures, and items) relevant to evaluation of eHealth implementation.
2. Create a database (compendium) of measures accessible to VA investigators and operations partners who are evaluating eHealth in both research projects and practice applications.
3. Complete a white paper to critically review existing measurement in the arena of eHealth for wide dissemination within the eHealth community.

METHODS:
With the assistance of a reference librarian at the University of Iowa Health Sciences Library, we conducted a comprehensive search of the literature. Databases included MEDLINE, Scopus, Health and Psychosocial Instruments, and PsychInfo for articles published in English through 2014. All search results were transferred to an Endnote database and duplicates were eliminated. Pairs of investigators reviewed titles of these articles to exclude ineligible articles (no instrument or use of an instrument), and remaining articles were retained for further review. The remaining article abstracts were reviewed independently by a pair of investigators. We limited our review to articles that reported at least one psychometric property, e.g., Cronbach's alpha; i.e., we did not include sets of "investigator developed" questions. Discrepancies were resolved by consensus. Remaining articles were then classified as describing the development and testing of a measure or using a measure. For articles using a measure, reference lists were hand searched to obtain citations for the original measure development.

FINDINGS/RESULTS:
The literature search resulted in 32,742 citations after duplicates were removed. Of those, 59 described an instrument, and 985 used an instrument. After reviewing abstracts, 26 measures/articles were included in the final data set.

IMPACT:
The VA has made a substantial investment in eHealth technologies to improve care for veterans. VA's ability to gauge such impacts hinges on the measurement tools that they have available and ultimately choose to use. The measures that we identified will be critical to initiatives across a number of operations offices, including the My HealtheVet Program Office, the Connected Health Office, the broader Office of Informatics and Analytics, the Office of Telehealth Services and even select resource centers like VIReC. The results of this RRP provide critical insights regarding existing eHealth measures and identify gaps where new metrics are needed. They can also inform future studies so that the results from multiple studies can be compared and synthesized because they used the same handful of metrics.

---

External Links for this Project

NIH Reporter

Grant Number: I21HX001218-01A1
Link: https://reporter.nih.gov/project-details/8659868

---

PUBLICATIONS:

Journal Articles

1. Wakefield BJ, Turvey CL, Nazi KM, Holman JE, Hogan TP, Shimada SL, Kennedy DR. Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis. Journal of medical Internet research. 2017 Oct 11; 19(10):e346. [view]

HSR&D or QUERI Articles

1. Wakefield BJ. The eHealth Measures Compendium. HSRD FORUM. 2014 May 1; 4. [view]

Conference Presentations

1. Wakefield BJ, Turvey CL, Hogan TP, Shimada S, Nazi K. Development of an eHealth Measures Compendium. Paper presented at: National Association for Home Care and Hospice Annual Home Healthcare and Hospice Information Technology Conference; 2014 Nov 15; Washington, DC. [view]

DRA: Health Systems
DRE: Research Infrastructure
Keywords: none
MeSH Terms: none