An online national quality assessment survey of prostate MRI reading: interreader variability in prostate volume measurement and PI-RADS classification

Background: High-quality assessment of prostate MRI is fundamental in both clinical practice and screening. There is a lack of national level data on variability in prostate volume measurement and PI-RADS assessment. Methods of quality assurance need to be developed. Methods: All Swedish radiology departments were invited to participate in an external quality assurance of prostate MRI reading. Ten prostate MRI cases were selected by an expert panel to reflect common findings. Readers measured whole gland volume (ellipsoid formula method) and assigned a PI-RADS score in a web-based PACS with full clinical functionality. Expert consensus was used as reference standard. Descriptive statistics were used to show the distribution of volume measurements and PSA density. Reader agreement was assessed using percentages and kappa scores. A feedback document was sent to all participants upon completion of the quality assurance program. Results: Forty-three radiologists representing 17 departments read at least 7 out of 10 cases. The median difference in prostate volume assessment compared to the reference volume for the 10 cases ranged from −23 mL to + 6 mL. Per case agreement ranged from 33 % to 86 % for the assigned PI-RADS score and from 35 % to 98 % for PI-RADS 1–3 versus PI-RADS 4–5. Interreader agreement was moderate with a median kappa score of 0.53 (IQR 0.48–0.62). Conclusion: This online model for national quality assurance programs was feasible. Rather large per-case reader variations in prostate volume assessment and PI-RADS scoring were shown. To reduce variability in clinical practice, systematic interreader comparisons should be encouraged.