To assess interobserver reliability and reproducibility of the Clinical Research Office of the Endourological Society (CROES) nephrolithometric nomogram.

Preoperative data and postoperative outcomes of 100 consecutive patients who underwent percutaneous nephrolithotomy were obtained. Patients' data were reviewed separately by 4 independent urologists of different academic level: an experienced attending endourologist, a graduated fellow, a young fellow, and a resident. Each rater adjusted a CROES score in all 100 patients. Interobserver reliability was analyzed by assessing intraclass correlation (ICC) and kappa coefficient among and between all different raters.

Assessment of interobserver reliability showed good or excellent agreement among all raters. Moderate agreement was only found between the raters for the “presence of staghorn” score. ICCs among all raters expressed excellent levels for each independent CROES parameter and reached great statistical significance. The highest correlation was noticed for “stone burden” in contrast to “presence of staghorn” parameter that showed the lowest. ICC for the final CROES score revealed good to excellent agreement among all raters and all pairs of raters.

CROES nephrolithometry is a reproducible nomogram. Reproducible results were obtained within a single institution by multiple reviewers of varying experience within a short period of time after undergoing standardized training.