Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Go to the ORD website
Go to the QUERI website

IIR 22-027 – HSR&D Study

 
IIR 22-027
Heterogeneity in Treatment Effect Timing in Geriatrics and Palliative Care Studies
Melissa Garrido,
Boston
Boston, MA
Funding Period: November 2022 - October 2026

Abstract

Background: In real-world studies of geriatric and palliative care programs, policies, or treatments, treatment initiation may need to be staggered across units in ways that are outside of investigators’ control. If differences across cohorts or in organizational characteristics associated with both treatment timing and outcome are not controlled for in analyses, they may obscure estimates of true treatment effects. Heterogeneous treatment timing is inadequately addressed in most existing methods. Current methods to account for treatment effect timing heterogeneity do not allow a treatment’s effect to be isolated from effects of confounders associated with both timing and outcomes. Significance: Staggered rollouts of policies and practice changes are common within VA — the timing at which a new intervention (e.g., Veteran Directed Care) is rolled out cannot always be controlled. Innovation and Impact: A potential solution involves inverse probability of treatment weights (IPTW) to adjust for confounding across treatment groups defined by receipt timing, but IPTWs lead to biased estimates in cross-sectional evaluations comparing multiple treatments. We have developed an alternative method, vector- based kernel weighting (VBKW), that outperforms IPTW in cross-sectional evaluations. The degree to which VBKW reduces bias and improves efficiency over IPTW in longitudinal applications has not yet been explored. Entropy balancing (EB) weights also may produce unbiased and efficient treatment effect estimates when combined with DD. Researchers need practical guidance for when weighting adjustments with DD may be a superior analytic method to account for treatment timing heterogeneity in a difference-in-differences study, or for when traditional two-way fixed effects (TWFE) approaches may be sufficient. Specific Aims: We aim to compare bias and efficiency of estimates using VBKW, IPTW, EB, and TWFE approaches in analyses of retrospective cohort studies with staggered treatment timing within a cohort (Aim 1) and on data from longitudinal panel studies where treatment effects may vary across and within cohorts of data collection (Aim 2). We will identify the degree of heterogeneity required for VBKW, IPTW, EB, and TWFE to lead to different inferences (Aim 3). Methodology: We will use Monte Carlo (MC) and plasmode simulations to evaluate bias, efficiency, covariate balance, and processing time for each strategy in data obtained from observational studies with staggered treatment timing. MC simulations on investigator-generated data (n=600, 900, 9600) will allow us to examine the impact of different analytic scenarios (e.g., sample distribution across treatment timing groups, dynamic effects) on the relative performance of estimators. Plasmode simulations will allow us to verify that our results are robust to data generating process and will be derived from an observational analysis of Veterans’ self- directed care services (using Corporate Data Warehouse data) and from the Health and Retirement Study (HRS). Next Steps/Implementation: We will identify when VBKW, EB, IPTW, or TWFE is superior for estimating the effect of a treatment provided at different times and summarize our results in practical guidance that we will share with the HSRD community. Our results will improve investigators’ ability to generate rigorous evidence from studies of Veterans’ geriatric and palliative care in which treatment or event timing cannot be controlled.

External Links for this Project

NIH Reporter

Grant Number: I01HX003602-01
Link: https://reporter.nih.gov/project-details/10533638



Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project

PUBLICATIONS:

None at this time.

DRA: None at this time.
DRE: None at this time.
Keywords: None at this time.
MeSH Terms: None at this time.

Questions about the HSR&D website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.