Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government
Veterans Benefits and Health Care About VA Find a VA Location

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

HSR Citation Abstract

Search | Search by Center | Search by Source | Keywords in Title

Are AI chatbots concordant with evidence-based cancer screening recommendations?

Nickel B, Ayre J, Marinovich ML, Smith DP, Chiam K, Lee CI, Wilt TJ, Taba M, McCaffery K, Houssami N. Are AI chatbots concordant with evidence-based cancer screening recommendations? Patient education and counseling. 2025 Jan 21; 134:108677.

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions


Abstract:

OBJECTIVE: This study aimed to assess whether information from AI chatbots on benefits and harms of breast and prostate cancer screening were concordant with evidence-based cancer screening recommendations. METHODS: Seven unique prompts (four breast cancer; three prostate cancer) were presented to ChatGPT in March 2024. A total of 60 criteria (30 breast; 30 prostate) were used to assess the concordance of information. Concordance was scored between 0 and 2 against the United States Preventive Services Task Force (USPSTF) breast and prostate cancer screening recommendations independently by international cancer screening experts. RESULTS: 43 of 60 (71.7?%) criteria were completely concordant, 3 (5?%) were moderately concordant and 14 (23.3?%) were not concordant or not present, with most of the non-concordant criteria (9 of 14, 64.3 %) being from prompts for the oldest age groups. ChatGPT hallucinations (i.e., completely made up, non-sensical or irrelevant information) were found in 9 of 60 criteria (15?%). CONCLUSIONS: ChatGPT provided information mostly concordant with USPSTF breast and prostate cancer screening recommendations, however, important gaps exist. These findings provide insights into the role of AI to communicate cancer screening benefits and harms and hold increased relevance for periods of guideline change. PRACTICE IMPLICATIONS: AI generated information on cancer screening should be taken in conjunction with official screening recommendations and/or information from clinicians.




Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.