- Open Access
Interobserver agreement of Prostate Imaging–Reporting and Data System (PI-RADS–v2)
Egyptian Journal of Radiology and Nuclear Medicine volume 52, Article number: 5 (2021)
A retrospective study was conducted on 71 consecutive patients with suspected prostate cancer (PCa) with a mean age of 56 years and underwent mp-MRI of the prostate at 3 Tesla MRI. Two readers recognized all prostatic lesions, and each lesion had a score according to Prostate Imaging–Reporting and Data System version 2 (PI-RADS-v2).
Purpose of the study
To evaluate the interobserver agreement of PI-RADS-v2 in characterization of prostatic lesions using multiparametric MRI (mp-MRI) at 3 Tesla MRI.
The overall interobserver agreement of PI-RADS-v2 for both zones was excellent (k = 0.81, percent agreement = 94.9%). In the peripheral zone (PZ) lesions are the interobserver agreement for PI-RADS II (k = 0.78, percent agreement = 83.9%), PI-RADS III (k = 0.66, percent agreement = 91.3 %), PI-RADS IV (k = 0.69, percent agreement = 93.5%), and PI-RADS V (k = 0.91, percent agreement = 95.7 %). In the transitional zone (TZ) lesions are the interobserver agreement for PI-RADS I (k = 0.98, percent of agreement = 96%), PI-RADS II (k = 0.65, percent agreement = 96%), PI-RADS III (k = 0.65, percent agreement = 88%), PI-RADS IV (k = 0.83, percent agreement = 96%), and PI-RADS V (k = 0.82, percent agreement = 92%).
We concluded that PI-RADS-v2 is a reliable and a reproducible imaging modality for the characterization of prostatic lesions and detection of PCa.
Prostate cancer (PCa) is the 2nd most frequent cancer in males with 5 years of survival rates getting about 99% endorsed by the early uncovering and better management procedures. The identification of PCa is unique going through a standard diagnostic path which includes a tans-rectal ultrasound (TRUS)-guided systematic sampling of the whole gland. Unfortunately, TRUS sampling may be affected by sampling error, with a remarkable Gleason score (GS) increase in the radical prostatectomy specimen. Such imperfection created an urge for a better pathway of early diagnosis. Fortunately, the technical advances have led to multi-parametric magnetic resonance imaging (mp-MRI) merging structural and functional MRI sequences [1,2,3,4,5,6,7,8].
Owing to the variability in MR machines, acquisition parameters, and individual assessment measures, the reporting of mp-MRI differs widely among radiologists. The American College of Radiology and European Society of Uroradiology created Prostate Imaging–Reporting and Data System version 1 (PI-RADS-v1) and its update version 2 (PI-RDAS-v2). The PI-RADS-v2 is created to increase the recognition, characterization, categorization, and risk stratification in patients with doubted malignancy. The main target is to build acceptable technical standards for prostate mp-MRI, which makes it clear and simpler with subsequent standardized reporting terms, as well as help the procedure of MRI-guided biopsy. It also develops assessment groups that summarize ranks of the risk that could be helpful to choose and prepare patients for the next steps in management and finally enhance interdisciplinary communications with physicians [6, 9,10,11,12,13,14,15,16,17,18,19]. Having both targeted and systematic biopsy could offer the peak discovery of clinically significant prostate cancer [20,21,22,23,24,25,26,27]. Numerous MRI-guided sampling procedures will enhance the recognition of clinically significant PCa as well as decrease the discovery of insignificant cancer decreasing the unnecessary costly treatments and their possible complications . Many studies discussed the interobserver agreement of PI-RADS-v2 in the assessment and discovery of prostate cancer [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. The uniqueness of our study is the assessment of interobserver in the peripheral zone (PZ) and the central zone (CZ) separately, as well as the interobserver agreement of the different pulse sequences of PI-RADS-v2.
Purpose of the study
The purpose of the study is to evaluate the interobserver agreement of PI-RADS-v2 in the characterization of prostatic lesions using mp-MRI at 3 Tesla MRI.
A retrospective, single-center study was permitted by the local research ethics committee of the hospitals, and the informed consent was waived because this is a retrospective study. During the period from April 2017 till January 2020, 95 male patients with clinically doubted PCa due to raised prostate-specific antigen (PSA) levels and/or atypical DRE were hired for our work. Twenty-four patients were omitted as follows: (1) the patients without pathological results (n = 9); (2) DCE imaging was not performed in the patient due to renal dysfunction and/or unwillingness to undergo the procedure (n = 10); and (3) poor quality of the MRI images due to movement artifacts, catheter artifacts, or the presence of hip implants (n = 5).
MP-MRI was performed at a 3T machine (Philips, USA) by a pelvic phased array surface coil. The examination technique includes high-resolution multi-planar T2WI by T2-weighted fast spin-echo imaging (TR/TE = 6000/102 ms, FOV = 140 mm; matrix = 256 × 192; intersection gap = 1 mm slice thickness, 3 mm. DWI at b values (0, 800, 1000, 1400 s/mm2), and its ADC map as follows: with free-breathing spin-echo EPI sequence (TR/TE = 3000/90 ms, slice thickness = ≤ 4 mm, no gap. FOV 16-22 cm, in-plane dimension: ≤ 2.5 mm phase and frequency. ADC maps developed from the least b value “50–100 s/mm2” and the highest 800–1400 s/mm2. Axial T1WI images formed through using a fast spin-echo sequence (TR/TE = 7.4/675 ms; [FOV] 140 mm; matrix size, 256 × 160; intersection gap, 1 mm; slice thickness 3 mm; the number of signals acquired, 2). Dynamic contrast-enhanced (DCE) T1 multi-planner images were achieved by IV injection of contrast (Dotarim (0.5 mmol/ml) with a quantity of 0.1 ml/kg body weight).
Image analysis was achieved by two uroradiologists (AH, MO), with 15 and 10 years of experience of prostate MR imaging not aware of the clinical findings and pathological diagnoses. First, the one uroradiologist with 15 years of experience identified and scored the suspicious lesions according to PI-RADS-V2 scoring. The same lesions were scored by another radiologist with 10 years of experience in another setting without the first radiologist. So, both readers were assigned and analyzed the same lesion. They reviewed the axial T1WI to exclude hemorrhage. T2-WI assessed the TZ to evaluate the presence of BPH or suspicious morphological changes. T2-WI for PZ was also essential to reveal the morphological feature for any suspected lesions. DWI was the cornerstone for PZ lesion and was searched to detect any suspicious bright signal on DWI or low signal at ADC. Also, TZ-suspected lesions are looked at if there is diffusion restriction or not. DCE mainly looked in it for already suspected lesions by T2 or DWI to reveal the presence of positive early enhancement and rapid washout. Lastly, a general look for the whole gland for abnormally non-homogenous enhanced. Each study was reported, and PI-RADS score from 1 to 5 was given for PZ according to DWI and for TZ according to T2-WI separately, and the overall PI-RADS score for each patient was given finally.
All prostatic lesion samples had been made by 10 years expert radiologists in the outpatient clinic by TRUS-guided biopsy of MRI doubtful prostatic lesions, and 10–12 systematic core prostate biopsies had been performed. Samples were fixed in formalin and stained with hematoxylin and eosin, then underwent comprehensive histopathologic assessment.
Analysis of data was performed by Statistical Package for Social Science version 20 (SPSS Inc., Chicago, Ill, USA). The interobserver agreement was expressed as a kappa (k) statistic with a 95% confidence interval (CI), and a p value < 0.05 was considered to indicate statistical significance. A қ is the amount of observed agreement. A қ of 0.0 represents an agreement that is equal to chance, a қ of 1.0 represents a perfect agreement, a қ of 0.81 to 1.0 is an excellent agreement, and 0.61 to 0.80 is a good agreement.
Seventy-one patients were included, with the final pathological diagnosis as follows: 53 patients diagnosed with PCa and 18 patients were diagnosed with benign hyperplasia tissue (n = 8), granulomatous prostatitis (n = 3), and prostatic abscesses (n = 7). The mean age of our study patients was 64.3 ± 8.5 years (range 46–88 years). Table 1 shows the kappa agreement of both observers for each zone. The overall interobserver agreement for both zones was excellent (k = 0.81, percent agreement = 94.9%).
Lesions at the PZ were reported in 46 patients by both observers, PI-RADS score II (Fig. 1) was reported in 5 patients (10.9%) by both observers with an excellent interobserver agreement (k = 0.78, p = 0.001), and percent of the agreement was 83.9%. PI-RADS score III (Fig. 2) was reported in 6 patients (13%) by observer one and in 8 patients (17.4%) by observer 2 with a fair agreement (k = 0.66), and the percent of the agreement was 91.3%. PI-RADS score IV was reported in 6 patients (13%) by observer 1 and in 5 patients (10.9%) by observer 2 with a good agreement (k = 0.69), and the percent of the agreement was 93.5%. PI-RADS score V (Fig. 3) was reported in 29 patients (63%) by observer 1 and in 28 patients (60.9%) by observer 2 with an excellent agreement (k = 0.91), and the percent of the agreement was 95.7%.
Lesions at the TZ were reported in 25 patients by both observers. The TZ lesions of PI-RADS I was reported in 7 patients (28%) by observer one and in 6 patients (24%) by observer 2 with an excellent interobserver agreement (k = 0.89, P = 0.001), and the percent of the agreement was 96.4%. PI-RADS II was reported in only one patient (4%) by observer one and in only two patients (8%) by observer 2 with an excellent agreement (k = 0.65), and the percent of the agreement was 96%. PI-RADS III was reported in 5 patients (20%) by observer 1 and in 6 patients (24%) by observer 2 with an excellent agreement (k = 0.65), and the percent of the agreement was 88%. PI-RADS IV was reported in 3 patients (12%) by observer 1 and in 4 patients (4%) by observer 2 with an excellent agreement (k = 0.83), and the percent of the agreement was 96%. PI-RADS V (Fig. 4) was reported in 9 patients (36%) by observer 1 and in 7 patients (28%) by observer 2 with an excellent agreement (k = 0.82), and the percent of agreement was 92%.
Table 2 shows the kappa agreement for different MR sequences. In PZ, features related to DWI reported in 46 patients (64.8%) by both observers with a good agreement (k = 0.78, percent of agreement = 86.96%). Features related to DCE for those lesion had an excellent agreement (k = 0.91, percent of agreement = 96%). In the TZ features related to lesion texture and margins on T2-WI reported in 25 patients (35.2%) by both observers with an excellent agreement (k = 0.95, percent of agreement = 96%). Features related to the DWI of TZ lesions were reported with an excellent agreement (k = 0.94, percent of agreement = 96%).
PI-RADS-v2 was released for the same language between the radiologist and the clinicians that used for early recognition of PCa [1, 3]. PI-RADS-v2 has a remarkable role in diagnosing PCa and provides a uniform protocol of mp-MRI, allowing a good range of interobserver agreement [20,21,22]. One study reported that good interobserver agreement rates use the most appropriate analysis (AC1 = 0.71) and moderate use kappa analysis (kappa = 0.43) . Few studies reported good interobserver agreement using PI-RADS V2 with a remarkable effect on the radiologist’s prior experience [19,20,21,22,23,24]. In our work, we found an excellent interobserver agreement using PI-RADS-v2 in reporting MP-MRI for prostate lesions. The difference in the results from the other studies may be attributed to the image analysis in our study which was done by two uroradiologists with a long time of experience compared to other studies which used general radiologists with a variable degree of experience.
In this study, there is an excellent interobserver agreement of both readers as they have a long period of experience of 15 and 10 years, respectively. Previous studies reported that the readers’ experience has an effect on the diagnostic performance of PI-RADS v2. The expert radiologists could recognize significant prostate cancer using PI-RADS-v2 with good agreement overall , and the agreement tended to be better in PZ than TZ, although was weak for DCE in PZ . There is a moderate agreement of PI-RADS of all categories of PCa (k = 0.53) and clinically significant cancers (csPCa) (k = 0.47) . One study reported that the interobserver agreement of PI-RADS is (k = 0.71) for both zones: for PZ (0.72) and for TZ (0.44) . Another study added that the overall interobserver agreement is 0.41 for PI-RADS score 3–5 and 0.51 for PI-RADS score 4–5 . The third study reported that interobserver agreement in PI-RADS v2 ranges from fair to good among radiologists and improves with increasing experience . The last study added that radiologists across experience levels had an excellent agreement for detecting index lesions and moderate agreement for category assignment of lesions using PI-RADS .
One study confirmed mp-MRI ability uncovering clinically significant PC with variability among radiologists , and another study added that PI-RADS-v2 had a moderate inter-reader agreement, with PI-RADS scores linking well with the possibility of intermediate- and high-grade cancers . However, a prior study referred to that it is restricted by an at-best moderate degree of agreement between readers .
The PZ lesion assessment depends mainly on DWI with a minor role for DCE [1,2,3]. Our work showed an excellent interobserver agreement for PZ lesions. Another study concluded a better interobserver agreement according to categories of the PZ than the TZ lesion .
Our study revealed a good interobserver agreement for TZ lesions. TZ lesion assessment depends mainly on T2-WI with a secondary role for DWI [5,6,7,8,9,10,11,12,13,14,15,16]. In a previous study, lesions of the PZ show good agreement regarding extra-prostatic extension and invasive behavior on T2-WI. The TZ lesions showed good agreement regarding EPE and moderate/marked hypointensity on T2-WI, while the corresponding positive or negative early enhancement at DCE had fair agreement .
PI-RADS-v2 follows the conception of “dominant sequence” as T2 is the hallmark for TZ and DWI is the hallmark for the PZ, with a minor role of the DCE [1,2,3,4,5]. In our study, there is an overall good agreement for PI-RADS in both the PZ and TZ. In addition, our study is unique as it assessed the interobserver agreement for each sequence by itself.
In PI-RADS-v2, T2WI is the cornerstone in the assessment of TZ lesions, with a minor role of the PZ lesion only to depict the abnormal morphological patterns [1,2,3,4,5]. One study reported that the interobserver agreement for T2-WI is 0.47 and 0.15 in the PZ and 0.37 and 0.07 in the TZ . In our study, T2-WI features of TZ lesions have been reported with an excellent interobserver agreement.
In our study, the DWI score of the PZ lesions revealed good agreement and that of TZ lesions revealed excellent agreement. DWI is used for assessment of oncology all over the body. One study reported that in PZ, reproducibility was moderate on DWI (κ = 0.535–0.619), fair on DCE (κ = 0.266–0.439), and fair for extraprostatic extension on T2-WI (κ = 0.289). In TZ, reproducibility for lesion texture and margins on T2-WI ranged from 0.136 (moderately hypointense) to 0.529 (encapsulation) . Another study added that encapsulated lenticular shape on T2WI, focal on DWI, and marked hypointensity on ADC map had a moderate agreement (K = 0.45 to 0.60), whereas heterogeneous and circumscribed on T2-WI, marked hyperintensity on high b value DWI, and the presence or not of early enhancement in the lesion/region of the lesion had a fair agreement (K = 0.30 to 0.38) .
In PIRDAS-v2, DCE changed from 5 points scoring to be rather −ve or +ve denoting lesser role than it was having in PI-RADS-v1 and different regions of the body. DCE is currently recognized as a second sequence in diagnosing PZ lesions. One study reported that the interobserver agreement of DCE is fair (k = 0.48–0.41) . In our work, features related to DCE for the PZ lesions were reported with an excellent agreement.
Our study has a few limitations. First, the reference standard was a TRUS-guided biopsy with a sampling error. Second, although this study was focused on lesion characterization according to the PI-RADSv2 assessment categories, it limited its ability to evaluate the accuracy of this method for lesion detection. Further multicenter studies are needed upon a large number of patients with calculation accuracy of Pi-RADS V2 in the detection of prostate lesions. Third, we applied PIRADS-v2 for the analysis of prostate lesions. We are recommending further studies with application advanced diffusion modules such as diffusion tensor imaging, MR spectroscopy, arterial spin labeling with machine learning, and whole-body imaging for staging of PCa.
We concluded that PI-RADS v2 is a reliable and reproducible imaging technique for the characterization of prostatic lesions and detection of PCa.
Availability of data and materials
The corresponding author is responsible for sending the used data and materials upon request.
Dynamic contrast enhancement
Multi-parametric magnetic resonance imaging
Prostate Imaging–Reporting and Data System
Padhani AR, Weinreb J, Rosenkrantz AB et al (2019) Prostate Imaging-Reporting and Data System Steering Committee: PI-RADS v2 status update and future directions. Eur Urol 75:385–396
Furlan A, Borhani AA, Westphalen AC (2018) Multiparametric MR imaging of the prostate: interpretation including Prostate Imaging Reporting and Data System version 2. Radiol Clin North Am 56:223–238
Steiger P, Thoeny HC (2016) Prostate MRI based on PI-RADS version 2: how we review and report. Cancer Imaging 16:9
Spektor M, Mathur M, Weinreb JC (2017) Standards for MRI reporting-the evolution to PI-RADS v 2.0. Transl Androl Urol 6:355–367
Jordan EJ, Fiske C, Zagoria RJ, Westphalen AC (2017) Evaluating the performance of PI-RADS v2 in the non-academic setting. Abdom Radiol 42:2725–2731
Wahab SA, Verma S (2016) Review of Prostate Imaging Reporting and Data System version 2. Future Oncol 12:2479–2494
Tempany C (2016) Opportunities for multiparametric MRI with PI-RADS v2 to make a difference. Future Oncol 12:2397–2399
Horn GL Jr, Hahn PF, Tabatabaei S et al (2016) A practical primer on PI-RADS version 2: a pictorial essay. Abdom Radiol 41:899–906
Hassanzadeh E, Glazer DI, Dunne RM et al (2017) Prostate Imaging Reporting and Data System version 2 (PI-RADS v2): a pictorial review. Abdom Radiol 42:278–289
Purysko AS, Rosenkrantz AB, Barentsz JO et al (2016) PI-RADS version 2: a pictorial update. Radiographics 36:1354–1372
Torregrosa Andrés A, Otero García M, Sineiro GM (2017) Magnetic resonance imaging of the prostate: interpretation using the PI-RADS V2. Radiologia 59:128–138
Turkbey B, Rosenkrantz AB, Haider MA et al (2019) Prostate Imaging Reporting and Data System version 2.1: 2019 update of Prostate Imaging Reporting and Data System version 2. Eur Urol 76:340–351
Schieda N (2020) Interobserver agreement of PI-RADS v. 2: not all features or observers are created equal. J Magn Reson Imaging 51:605–606
Mussi TC, Yamauchi FI, Tridente CF et al (2020) Interobserver agreement of PI-RADS v. 2 lexicon among radiologists with different levels of experience. J Magn Reson Imaging 51:593–602
Girometti R, Giannarini G, Greco F et al (2019) Interreader agreement of PI-RADS v. 2 in assessing prostate cancer with multiparametric MRI: a study using whole-mount histology as the standard of reference. J Magn Reson Imaging 49:546–555
Greer MD, Shih JH, Lay N et al (2019) Interreader variability of Prostate Imaging Reporting and Data System version 2 in detecting and assessing prostate cancer lesions at prostate MRI. AJR Am J Roentgenol. https://doi.org/10.2214/AJR.18.20536
Mussi TC, Yamauchi FI, Tridente CF et al (2019) Interobserver agreement and positivity of PI-RADS version 2 among radiologists with different levels of experience. Acad Radiol 26:1017–1022
Kohestani K, Wallström J, Dehlfors N et al (2019) Performance and interobserver variability of prostate MRI (PI-RADS version 2) outside high-volume centres. Scand J Urol 53:304–311
Ke Z, Wang L, Min XD et al (2018) Diagnostic performance and interobserver consistency of the Prostate Imaging Reporting and Data System version 2: a study on six prostate radiologists with different experiences from half a year to 17 years. Chin Med J 131:1666–1673
Popita C, Popita AR, Andrei A et al (2018) Interobserver agreement in prostate cancer detection using multiparametric MRI. J BUON 23:1061–1069
Hofbauer SL, Maxeiner A, Kittner B et al (2018) Validation of Prostate Imaging Reporting and Data System version 2 for the detection of prostate cancer. J Urol 200:767–773
Kim SH, Choi MS, Kim MJ et al (2017) Validation of Prostate Imaging Reporting and Data System version 2 using an mri-ultrasound fusion biopsy in prostate cancer diagnosis. AJR Am J Roentgenol 209:800–805
Glazer DI, Mayo-Smith WW, Sainani NI et al (2017) Interreader agreement of Prostate Imaging Reporting and Data System version 2 using an in-bore MRI-guided prostate biopsy cohort: a single institution’s initial experience. AJR Am J Roentgenol 209:W145–W151
Flood TF, Pokharel SS, Patel NU et al (2017) Accuracy and interobserver variability in reporting of pi-rads version 2. J Am Coll Radiol 14:1202–1205
Nguyentat M, Ushinsky A, Miranda-Aguirre A et al (2018) Validation of Prostate Imaging-Reporting and Data System version 2: a retrospective analysis. Curr Probl Diagn Radiol 47:404–409
Baldisserotto M, Neto EJ, Carvalhal G et al (2016) Validation of PI-RADS v.2 for prostate cancer diagnosis with MRI at 3T using an external phased-array coil. J Magn Reson Imaging 44:1354–1359
Purysko AS, Bittencourt LK, Bullen JA et al (2017) Accuracy and interobserver agreement for Prostate Imaging Reporting and Data System, version 2, for the characterization of lesions identified on multiparametric MRI of the prostate. AJR Am J Roentgenol 209:339–349
Chen F, Cen S, Palmer S (2017) Application of Prostate Imaging Reporting and Data System version 2 (PI-RADS v2): interobserver agreement and positive predictive value for localization of intermediate- and high-grade prostate cancers on multiparametric magnetic resonance imaging. Acad Radiol 24:1101–1106
Rosenkrantz AB, Ginocchio LA, Cornfeld D et al (2016) Interobserver reproducibility of the PI-RADS Version 2 lexicon: a multicenter study of six experienced prostate radiologists. Radiology 280:793–804
Muller BG, Shih JH, Sankineni S et al (2015) Prostate cancer: interobserver agreement and accuracy with the revised Prostate Imaging Reporting and Data System at multiparametric mr imaging. Radiology 277:741–750
Thai JN, Narayanan HA, George AK et al (2018) Validation of PI-RADS version 2 in transition zone lesions for the detection of prostate cancer. Radiology 288:485–491
No source of funding.
Ethics approval and consent to participate
The study was approved by the ethical committee of the radiology and the urology departments of a highly specialized academic hospital, and an informed written consent was taken from all patients that were included in the study. The ethics committee reference number is Ref. No. aswu /126/4/17.
Consent for publication
All authors approved the manuscript. All patients included in this research were legible. They gave written informed consent to publish the data contained within this study.
The authors declare that they have no competing interests.
About this article
Cite this article
Ahmed, H.M., Ebeed, A.E., Hamdy, A. et al. Interobserver agreement of Prostate Imaging–Reporting and Data System (PI-RADS–v2). Egypt J Radiol Nucl Med 52, 5 (2021). https://doi.org/10.1186/s43055-020-00378-w
- Magnetic resonance imaging