Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

HSR&D Citation Abstract

Search | Search by Center | Search by Source | Keywords in Title

Can raters consistently evaluate the content of focus groups?

Weinberger M, Ferguson JA, Westmoreland G, Mamlin LA, Segar DS, Eckert GJ, Greene JY, Martin DK, Tierney WM. Can raters consistently evaluate the content of focus groups? Social science & medicine (1982). 1998 Apr 1; 46(7):929-33.

Related HSR&D Project(s)

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
   Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions


Focus groups are increasingly being used to provide insights to researchers and policy makers. These data complement quantitative approaches to understanding the world. Unfortunately, quantitative and qualitative methodologies have often been viewed as antithetical, rather than complementary, strategies. While focus groups can clearly generate rich information that is unobtainable through other quantitative methods, it is important to determine the degree to which different raters can consistently extract information from transcripts. Thus, our goal was to quantify agreement in the interpretation of transcripts from patient and physician focus groups, using decision-making in ischemic heart disease as a model. We used data from focus groups with both patients and physicians that sought to identify factors affecting diagnostic and treatment decisions in ischemic heart disease. Three raters independently reviewed transcribed audiotapes from focus groups of patients with ischemic heart disease, as well as focus groups of physicians who care for these patients. We found that raters could not distinguish between major and minor factors reliably. More troubling, however, is that consistency regarding the apparently straightforward judgment as to the mere presence or absence of a factor was difficult to achieve. In particular, the three raters of each transcript failed to agree on between one third and one half of the factors. This reasonably high level of disagreement occurred despite the raters: (1) having generated the individual factors themselves based upon their reading a random sample of actual transcripts and (2) being trained in the use of rating forms (including standard definitions of themes). These data suggest that if a single rater evaluates focus group transcripts, as is commonly done, judgments may not be reproducible by other raters. Moreover, a single rater may not extract all important information contained in the transcripts.

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.