Interobserver agreement of Prostate Imaging–Reporting and Data System (PI-RADS–v2)

Abstract

Background

A retrospective study was conducted on 71 consecutive patients with suspected prostate cancer (PCa) with a mean age of 56 years and underwent mp-MRI of the prostate at 3 Tesla MRI. Two readers recognized all prostatic lesions, and each lesion had a score according to Prostate Imaging–Reporting and Data System version 2 (PI-RADS-v2).

Purpose of the study

To evaluate the interobserver agreement of PI-RADS-v2 in characterization of prostatic lesions using multiparametric MRI (mp-MRI) at 3 Tesla MRI.

Results

The overall interobserver agreement of PI-RADS-v2 for both zones was excellent (k = 0.81, percent agreement = 94.9%). In the peripheral zone (PZ) lesions are the interobserver agreement for PI-RADS II (k = 0.78, percent agreement = 83.9%), PI-RADS III (k = 0.66, percent agreement = 91.3 %), PI-RADS IV (k = 0.69, percent agreement = 93.5%), and PI-RADS V (k = 0.91, percent agreement = 95.7 %). In the transitional zone (TZ) lesions are the interobserver agreement for PI-RADS I (k = 0.98, percent of agreement = 96%), PI-RADS II (k = 0.65, percent agreement = 96%), PI-RADS III (k = 0.65, percent agreement = 88%), PI-RADS IV (k = 0.83, percent agreement = 96%), and PI-RADS V (k = 0.82, percent agreement = 92%).

Conclusion

We concluded that PI-RADS-v2 is a reliable and a reproducible imaging modality for the characterization of prostatic lesions and detection of PCa.

Background

Prostate cancer (PCa) is the 2nd most frequent cancer in males with 5 years of survival rates getting about 99% endorsed by the early uncovering and better management procedures. The identification of PCa is unique going through a standard diagnostic path which includes a tans-rectal ultrasound (TRUS)-guided systematic sampling of the whole gland. Unfortunately, TRUS sampling may be affected by sampling error, with a remarkable Gleason score (GS) increase in the radical prostatectomy specimen. Such imperfection created an urge for a better pathway of early diagnosis. Fortunately, the technical advances have led to multi-parametric magnetic resonance imaging (mp-MRI) merging structural and functional MRI sequences [1,2,3,4,5,6,7,8].

Owing to the variability in MR machines, acquisition parameters, and individual assessment measures, the reporting of mp-MRI differs widely among radiologists. The American College of Radiology and European Society of Uroradiology created Prostate Imaging–Reporting and Data System version 1 (PI-RADS-v1) and its update version 2 (PI-RDAS-v2). The PI-RADS-v2 is created to increase the recognition, characterization, categorization, and risk stratification in patients with doubted malignancy. The main target is to build acceptable technical standards for prostate mp-MRI, which makes it clear and simpler with subsequent standardized reporting terms, as well as help the procedure of MRI-guided biopsy. It also develops assessment groups that summarize ranks of the risk that could be helpful to choose and prepare patients for the next steps in management and finally enhance interdisciplinary communications with physicians [6, 9,10,11,12,13,14,15,16,17,18,19]. Having both targeted and systematic biopsy could offer the peak discovery of clinically significant prostate cancer [20,21,22,23,24,25,26,27]. Numerous MRI-guided sampling procedures will enhance the recognition of clinically significant PCa as well as decrease the discovery of insignificant cancer decreasing the unnecessary costly treatments and their possible complications [27]. Many studies discussed the interobserver agreement of PI-RADS-v2 in the assessment and discovery of prostate cancer [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. The uniqueness of our study is the assessment of interobserver in the peripheral zone (PZ) and the central zone (CZ) separately, as well as the interobserver agreement of the different pulse sequences of PI-RADS-v2.

Purpose of the study

The purpose of the study is to evaluate the interobserver agreement of PI-RADS-v2 in the characterization of prostatic lesions using mp-MRI at 3 Tesla MRI.

Methods

Study population

A retrospective, single-center study was permitted by the local research ethics committee of the hospitals, and the informed consent was waived because this is a retrospective study. During the period from April 2017 till January 2020, 95 male patients with clinically doubted PCa due to raised prostate-specific antigen (PSA) levels and/or atypical DRE were hired for our work. Twenty-four patients were omitted as follows: (1) the patients without pathological results (n = 9); (2) DCE imaging was not performed in the patient due to renal dysfunction and/or unwillingness to undergo the procedure (n = 10); and (3) poor quality of the MRI images due to movement artifacts, catheter artifacts, or the presence of hip implants (n = 5).

MR imaging

MP-MRI was performed at a 3T machine (Philips, USA) by a pelvic phased array surface coil. The examination technique includes high-resolution multi-planar T2WI by T2-weighted fast spin-echo imaging (TR/TE = 6000/102 ms, FOV = 140 mm; matrix = 256 × 192; intersection gap = 1 mm slice thickness, 3 mm. DWI at b values (0, 800, 1000, 1400 s/mm²), and its ADC map as follows: with free-breathing spin-echo EPI sequence (TR/TE = 3000/90 ms, slice thickness = ≤ 4 mm, no gap. FOV 16-22 cm, in-plane dimension: ≤ 2.5 mm phase and frequency. ADC maps developed from the least b value “50–100 s/mm²” and the highest 800–1400 s/mm². Axial T1WI images formed through using a fast spin-echo sequence (TR/TE = 7.4/675 ms; [FOV] 140 mm; matrix size, 256 × 160; intersection gap, 1 mm; slice thickness 3 mm; the number of signals acquired, 2). Dynamic contrast-enhanced (DCE) T1 multi-planner images were achieved by IV injection of contrast (Dotarim (0.5 mmol/ml) with a quantity of 0.1 ml/kg body weight).

Image analysis

Image analysis was achieved by two uroradiologists (AH, MO), with 15 and 10 years of experience of prostate MR imaging not aware of the clinical findings and pathological diagnoses. First, the one uroradiologist with 15 years of experience identified and scored the suspicious lesions according to PI-RADS-V2 scoring. The same lesions were scored by another radiologist with 10 years of experience in another setting without the first radiologist. So, both readers were assigned and analyzed the same lesion. They reviewed the axial T1WI to exclude hemorrhage. T2-WI assessed the TZ to evaluate the presence of BPH or suspicious morphological changes. T2-WI for PZ was also essential to reveal the morphological feature for any suspected lesions. DWI was the cornerstone for PZ lesion and was searched to detect any suspicious bright signal on DWI or low signal at ADC. Also, TZ-suspected lesions are looked at if there is diffusion restriction or not. DCE mainly looked in it for already suspected lesions by T2 or DWI to reveal the presence of positive early enhancement and rapid washout. Lastly, a general look for the whole gland for abnormally non-homogenous enhanced. Each study was reported, and PI-RADS score from 1 to 5 was given for PZ according to DWI and for TZ according to T2-WI separately, and the overall PI-RADS score for each patient was given finally.

Pathologic analysis

All prostatic lesion samples had been made by 10 years expert radiologists in the outpatient clinic by TRUS-guided biopsy of MRI doubtful prostatic lesions, and 10–12 systematic core prostate biopsies had been performed. Samples were fixed in formalin and stained with hematoxylin and eosin, then underwent comprehensive histopathologic assessment.

Statistical analysis

Analysis of data was performed by Statistical Package for Social Science version 20 (SPSS Inc., Chicago, Ill, USA). The interobserver agreement was expressed as a kappa (k) statistic with a 95% confidence interval (CI), and a p value < 0.05 was considered to indicate statistical significance. A қ is the amount of observed agreement. A қ of 0.0 represents an agreement that is equal to chance, a қ of 1.0 represents a perfect agreement, a қ of 0.81 to 1.0 is an excellent agreement, and 0.61 to 0.80 is a good agreement.

Results

Seventy-one patients were included, with the final pathological diagnosis as follows: 53 patients diagnosed with PCa and 18 patients were diagnosed with benign hyperplasia tissue (n = 8), granulomatous prostatitis (n = 3), and prostatic abscesses (n = 7). The mean age of our study patients was 64.3 ± 8.5 years (range 46–88 years). Table 1 shows the kappa agreement of both observers for each zone. The overall interobserver agreement for both zones was excellent (k = 0.81, percent agreement = 94.9%).

Table 1 Interobserver agreement of PZ and TZ of PI-RARDS-v2

Lesions at the PZ were reported in 46 patients by both observers, PI-RADS score II (Fig. 1) was reported in 5 patients (10.9%) by both observers with an excellent interobserver agreement (k = 0.78, p = 0.001), and percent of the agreement was 83.9%. PI-RADS score III (Fig. 2) was reported in 6 patients (13%) by observer one and in 8 patients (17.4%) by observer 2 with a fair agreement (k = 0.66), and the percent of the agreement was 91.3%. PI-RADS score IV was reported in 6 patients (13%) by observer 1 and in 5 patients (10.9%) by observer 2 with a good agreement (k = 0.69), and the percent of the agreement was 93.5%. PI-RADS score V (Fig. 3) was reported in 29 patients (63%) by observer 1 and in 28 patients (60.9%) by observer 2 with an excellent agreement (k = 0.91), and the percent of the agreement was 95.7%.

Fig. 1

PI-RADS II (benign prostatic hyperplasia). a Axial T2-WI shows TZ circumscribed hypointense encapsulated nodules with unremarkable PZ signal. b DWI shows no restricted diffusion. c ADC map shows no suspected areas of low signal

Fig. 2

PI-RADS III (prostate cancer). a Axial T2-WI shows TZ heterogeneous signal intensity with obscured margins. b Axial DWI shows no restricted diffusion. c Axial ADC map shows focal mild hypointensity. d Axial contrast MR image shows homogenous strong +VE uptake of the TZ adenomas

Fig. 3

PI-RADS V of PZ (prostate cancer). a Axial T2-WI shows right PZ well-defined focal abnormal hypointense lesion with extra-capsular extension as well as invading the right neurovascular bundle. b DWI shows corresponding focal marked hyperintensity > 1.5 cm. c ADC map shows corresponding focal marked hypointensity. d Axial contrast MR image shows corresponding +VE uptake

Lesions at the TZ were reported in 25 patients by both observers. The TZ lesions of PI-RADS I was reported in 7 patients (28%) by observer one and in 6 patients (24%) by observer 2 with an excellent interobserver agreement (k = 0.89, P = 0.001), and the percent of the agreement was 96.4%. PI-RADS II was reported in only one patient (4%) by observer one and in only two patients (8%) by observer 2 with an excellent agreement (k = 0.65), and the percent of the agreement was 96%. PI-RADS III was reported in 5 patients (20%) by observer 1 and in 6 patients (24%) by observer 2 with an excellent agreement (k = 0.65), and the percent of the agreement was 88%. PI-RADS IV was reported in 3 patients (12%) by observer 1 and in 4 patients (4%) by observer 2 with an excellent agreement (k = 0.83), and the percent of the agreement was 96%. PI-RADS V (Fig. 4) was reported in 9 patients (36%) by observer 1 and in 7 patients (28%) by observer 2 with an excellent agreement (k = 0.82), and the percent of agreement was 92%.

Fig. 4

PI-RADS V (prostate cancer). a Axial T2-WI shows extensive mass involves the whole gland with extra-capsular extension as well as invading both neurovascular bundles. b DWI shows corresponding focal marked hyperintensity > 1.5 cm. c ADC map shows corresponding focal marked hypointensity. d Axial contrast MR image shows corresponding +VE uptake

Table 2 shows the kappa agreement for different MR sequences. In PZ, features related to DWI reported in 46 patients (64.8%) by both observers with a good agreement (k = 0.78, percent of agreement = 86.96%). Features related to DCE for those lesion had an excellent agreement (k = 0.91, percent of agreement = 96%). In the TZ features related to lesion texture and margins on T2-WI reported in 25 patients (35.2%) by both observers with an excellent agreement (k = 0.95, percent of agreement = 96%). Features related to the DWI of TZ lesions were reported with an excellent agreement (k = 0.94, percent of agreement = 96%).

Table 2 Interobserver agreement of MR sequences of PI-RARDS-v2

Discussion

PI-RADS-v2 was released for the same language between the radiologist and the clinicians that used for early recognition of PCa [1, 3]. PI-RADS-v2 has a remarkable role in diagnosing PCa and provides a uniform protocol of mp-MRI, allowing a good range of interobserver agreement [20,21,22]. One study reported that good interobserver agreement rates use the most appropriate analysis (AC1 = 0.71) and moderate use kappa analysis (kappa = 0.43) [17]. Few studies reported good interobserver agreement using PI-RADS V2 with a remarkable effect on the radiologist’s prior experience [19,20,21,22,23,24]. In our work, we found an excellent interobserver agreement using PI-RADS-v2 in reporting MP-MRI for prostate lesions. The difference in the results from the other studies may be attributed to the image analysis in our study which was done by two uroradiologists with a long time of experience compared to other studies which used general radiologists with a variable degree of experience.

In this study, there is an excellent interobserver agreement of both readers as they have a long period of experience of 15 and 10 years, respectively. Previous studies reported that the readers’ experience has an effect on the diagnostic performance of PI-RADS v2. The expert radiologists could recognize significant prostate cancer using PI-RADS-v2 with good agreement overall [27], and the agreement tended to be better in PZ than TZ, although was weak for DCE in PZ [29]. There is a moderate agreement of PI-RADS of all categories of PCa (k = 0.53) and clinically significant cancers (csPCa) (k = 0.47) [15]. One study reported that the interobserver agreement of PI-RADS is (k = 0.71) for both zones: for PZ (0.72) and for TZ (0.44) [17]. Another study added that the overall interobserver agreement is 0.41 for PI-RADS score 3–5 and 0.51 for PI-RADS score 4–5 [18]. The third study reported that interobserver agreement in PI-RADS v2 ranges from fair to good among radiologists and improves with increasing experience [14]. The last study added that radiologists across experience levels had an excellent agreement for detecting index lesions and moderate agreement for category assignment of lesions using PI-RADS [16].

One study confirmed mp-MRI ability uncovering clinically significant PC with variability among radiologists [18], and another study added that PI-RADS-v2 had a moderate inter-reader agreement, with PI-RADS scores linking well with the possibility of intermediate- and high-grade cancers [28]. However, a prior study referred to that it is restricted by an at-best moderate degree of agreement between readers [13].

The PZ lesion assessment depends mainly on DWI with a minor role for DCE [1,2,3]. Our work showed an excellent interobserver agreement for PZ lesions. Another study concluded a better interobserver agreement according to categories of the PZ than the TZ lesion [29].

Our study revealed a good interobserver agreement for TZ lesions. TZ lesion assessment depends mainly on T2-WI with a secondary role for DWI [5,6,7,8,9,10,11,12,13,14,15,16]. In a previous study, lesions of the PZ show good agreement regarding extra-prostatic extension and invasive behavior on T2-WI. The TZ lesions showed good agreement regarding EPE and moderate/marked hypointensity on T2-WI, while the corresponding positive or negative early enhancement at DCE had fair agreement [14].

PI-RADS-v2 follows the conception of “dominant sequence” as T2 is the hallmark for TZ and DWI is the hallmark for the PZ, with a minor role of the DCE [1,2,3,4,5]. In our study, there is an overall good agreement for PI-RADS in both the PZ and TZ. In addition, our study is unique as it assessed the interobserver agreement for each sequence by itself.

In PI-RADS-v2, T2WI is the cornerstone in the assessment of TZ lesions, with a minor role of the PZ lesion only to depict the abnormal morphological patterns [1,2,3,4,5]. One study reported that the interobserver agreement for T2-WI is 0.47 and 0.15 in the PZ and 0.37 and 0.07 in the TZ [30]. In our study, T2-WI features of TZ lesions have been reported with an excellent interobserver agreement.

In our study, the DWI score of the PZ lesions revealed good agreement and that of TZ lesions revealed excellent agreement. DWI is used for assessment of oncology all over the body. One study reported that in PZ, reproducibility was moderate on DWI (κ = 0.535–0.619), fair on DCE (κ = 0.266–0.439), and fair for extraprostatic extension on T2-WI (κ = 0.289). In TZ, reproducibility for lesion texture and margins on T2-WI ranged from 0.136 (moderately hypointense) to 0.529 (encapsulation) [29]. Another study added that encapsulated lenticular shape on T2WI, focal on DWI, and marked hypointensity on ADC map had a moderate agreement (K = 0.45 to 0.60), whereas heterogeneous and circumscribed on T2-WI, marked hyperintensity on high b value DWI, and the presence or not of early enhancement in the lesion/region of the lesion had a fair agreement (K = 0.30 to 0.38) [14].

In PIRDAS-v2, DCE changed from 5 points scoring to be rather −ve or +ve denoting lesser role than it was having in PI-RADS-v1 and different regions of the body. DCE is currently recognized as a second sequence in diagnosing PZ lesions. One study reported that the interobserver agreement of DCE is fair (k = 0.48–0.41) [29]. In our work, features related to DCE for the PZ lesions were reported with an excellent agreement.

Our study has a few limitations. First, the reference standard was a TRUS-guided biopsy with a sampling error. Second, although this study was focused on lesion characterization according to the PI-RADSv2 assessment categories, it limited its ability to evaluate the accuracy of this method for lesion detection. Further multicenter studies are needed upon a large number of patients with calculation accuracy of Pi-RADS V2 in the detection of prostate lesions. Third, we applied PIRADS-v2 for the analysis of prostate lesions. We are recommending further studies with application advanced diffusion modules such as diffusion tensor imaging, MR spectroscopy, arterial spin labeling with machine learning, and whole-body imaging for staging of PCa.

Conclusion

We concluded that PI-RADS v2 is a reliable and reproducible imaging technique for the characterization of prostatic lesions and detection of PCa.