Measurement of the Inter-Rater Reliability Rate Is Mandatory for Improving the Quality of a Medical Database: Experience with the Paulista Lung Cancer Registry - 24/05/18
, Priscila B. Costa, RN a, Michele Salati, MD, PhD d, Paulo M. Pego-Fernandes, MD, PhD a, c, Ricardo M. Terra, MD, PhD a, bAbstract |
Background |
Database quality measurement should be considered a mandatory step to ensure an adequate level of confidence in data used for research and quality improvement. Several metrics have been described in the literature, but no standardized approach has been established. We aimed to describe a methodological approach applied to measure the quality and inter-rater reliability of a regional multicentric thoracic surgical database (Paulista Lung Cancer Registry).
Study Design |
Data from the first 3 years of the Paulista Lung Cancer Registry underwent an audit process with 3 metrics: completeness, consistency, and inter-rater reliability. The first 2 methods were applied to the whole data set, and the last method was calculated using 100 cases randomized for direct auditing. Inter-rater reliability was evaluated using percentage of agreement between the data collector and auditor and through calculation of Cohen's κ and intraclass correlation.
Results |
The overall completeness per section ranged from 0.88 to 1.00, and the overall consistency was 0.96. Inter-rater reliability showed many variables with high disagreement (>10%). For numerical variables, intraclass correlation was a better metric than inter-rater reliability. Cohen's κ showed that most variables had moderate to substantial agreement.
Conclusions |
The methodological approach applied to the Paulista Lung Cancer Registry showed that completeness and consistency metrics did not sufficiently reflect the real quality status of a database. The inter-rater reliability associated with κ and intraclass correlation was a better quality metric than completeness and consistency metrics because it could determine the reliability of specific variables used in research or benchmark reports. This report can be a paradigm for future studies of data quality measurement.
Le texte complet de cet article est disponible en PDF.Plan
| Disclosure Information: Nothing to disclose. |
|
| Disclosures outside the scope of this work: Dr Terra is a member of the Johnson & Johnson and Pfizer Advisory Boards; receives payment for lectures from Medtronic, Boehringer, and MSD; and receives payment for development of educational presentations from H Strattner/Intuitive. |
|
| Support for this study: This work was supported by the São Paulo Research Foundation (FAPESP) (protocol number: 2012/51764-8). |
Vol 226 - N° 6
P. 1128-1136 - juin 2018 Retour au numéro
Article précédent
- Implementing a Novel Guideline to Prevent Hospital-Acquired Pressure Ulcers in a Trauma Population: A Patient-Safety Approach
- Crystal Lam, Adel Elkbuli, Brenda Benson, Elizabeth Young, Orlando Morejon, Dessy Boneva, Shaikh Hai, Mark McKenney
- External Validation and Optimization of the French Association of Hepatopancreatobiliary Surgery and Transplantation's Score to Predict Severe Postoperative Biliary Leakage after Open or Laparoscopic Liver Resection
- Kayvan Mohkam, David Fuks, Eric Vibert, Takeo Nomi, François Cauchy, Yoshikuni Kawaguchi, Emmanuel Boleslawski, Jean-Marc Regimbeau, Brice Gayet, Jean-Yves Mabrut
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?