What a catch! traits that define good annotators.

Forbush TB, Shen S, South BR, Duvalla SL. What a catch! traits that define good annotators. Studies in health technology and informatics. 2013 Jan 1; 192:1213.

Related HSR&D Project(s)

• HIR 09-001 – Consortium of Healthcare Informatics Research: Document Quality and Challenge Evaluations

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

Human annotation and chart review is an important process in biomedical informatics research, but which humans are best suited for the job? Domain expertise, such as medical or linguistic knowledge is desirable, but other factors may be equally important. The University of Utah has a group of 20+ reviewers with backgrounds in medicine and linguistics, and 10 key traits have surfaced in those best able to annotate quickly and with high quality. To identify reviewers with these key traits, we created a hiring process that includes interviewing candidates, testing their medical and linguistic knowledge, and having them complete an annotation exercise on realistic medical text. Each step is designed to assess the key traits and allow the investigator to choose the skill set required for each project.