A detailed statistical analysis of the performance of CO-RADS and CT-severity score in the diagnosis of COVID-19 pneumonia compared to RT-PCR test: a prospective cohort study

Reports from international studies regarding the role of CT scan and RT-PCR test in the diagnosis of coronavirus disease has been a subject of controversy. The purpose of this study was to statistically compare the performance of CT in reporting chest CT scans of coronavirus disease according to Coronavirus Disease Reporting and Data System (CO-RADS) and CT severity score (CTSS) with the performance of RT-PCR test. The analyzed CT scans of 144 participants were consistent with CO-RADS 1 (n = 38), CO-RADS 2 (n = 11), CO-RADS 3 (n = 35), CO-RADS 4 (n = 23), and CO-RADS 5 (n = 37). CTSS in CO-RADS 1 was (0.9 ± 4), CO-RADS 2 (4 ± 2), CO-RADS 3 (10.2 ± 2), CO-RADS 4 (14 ± 6) and CO-RADS 5 (19 ± 7). There was direct correlation between CO-RADS groups and CTSS (p < 0.001). The mean total CTSS was 10 ± 9 for the whole study population. Ninety-five CT scans were compatible with CO-RADS 3, 4 or 5 and 49 CT scans were compatible with CO-RADS 1 or 2, with a positive rate of 66% (95% CI 49%, 65%), PPV (55.41%), NPV (45.18%), accuracy (86.8%) and the overall sensitivity (93.18%) and specificity (76.8%) of CT in detecting COVID-19 pneumonia when categorized and analyzed according to CO-RADS and CTSS. Sixty-four patients had positive initial RT-PCR tests and 80 patients had negative initial RT-PCR test, with a positive rate of 44.4% (95% CI 35%, 51%), PPV (41.13%), NPV (59.51%), accuracy (74.3%), sensitivity (64.2%) and specificity (93.9%). The Kappa (κ) value of average inter-reader agreement was 88% (95% CI 80%, 96%). RT-PCR test showed higher specificity and NPV compared to CT in detecting COVID-19 pneumonia, while CT showed higher sensitivity, PPV, accuracy and positive rate, respectively. CT was superior to RT-PCR test in detecting COVID-19 pneumonia especially at early stages of the disease.

with pharyngeal swab or bronchoalveolar lavage [3][4][5].However, its reported sensitivities vary from 42 to 83%, and it can take several days to obtain [3][4][5][6][7].CT plays a central role in the diagnosis of COVID-19 pneumonia, which is suspected when ground-glass opacities (GGO) with or without consolidations are present, yet CT is not considered as a definitive diagnostic tool as a positive RT-PCR test [7].Whether CT or RT-PCR test is more reliable in the diagnosis of COVID-19 pneumonia, has been debated in previous studies [3][4][5][6][7].The Dutch Radiological Society has developed a COVID-19 pneumonia reporting and data system (CO-RADS) to categorize chest CT scans with pulmonary changes that can be compatible with the COVID-19 pneumonia [8].According to findings on CT, graded from CO-RADS 1 to CO-RADS 5, COVID-19 pneumonia is unlikely in CO-RADS-1 and highly suspected in CO-RADS 5.In Addition, several studies encouraged the use of a CT severity score (CTSS) combined with CO-RADS to estimate the degree of lung involvement [9][10][11][12][13][14]. Consensus on standardized radiology reports combined with clinical assessment of symptoms may justify further management in case of negative (RT-PCR) test [7].
The purpose of this study was to statistically compare the performance of CT as analyzed according to CO-RADS and CTSS, with the performance of RT-PCR in detecting COVID-19 pneumonia.As far as we know, this is one the few studies (if any) providing a detailed statistical analysis of both CT and RT-PCR test in detecting coronavirus disease including sensitivity, specificity, positive predictive value, negative predictive value, accuracy and positive rate for each test.

Study population
After obtaining approval from the ethical authorities, this prospective cohort study was conducted in 283 consecutive patients admitted to our hospitals between October 2021 and May 2022.The patients' characteristics and comorbidities appear in Tables 1 and 2, respectively.Inclusion criteria included: fever, respiratory and or gastrointestinal symptoms, imaging features of pneumonia (Table 3), abnormal infection parameters including high CRP (> 8 mg/l), and lymphocytopenia (< 1.3 10^9/l) (Table 4), contact with SARS-CoV-2 patients in the last 14 days including travel history to epidemic region and had at least one valid RT-PCR test performed at least 5 days after symptoms debut.
Patients were excluded due to: known parenchymal lunge disease or malignancy (n = 35), lack of CT scan (n = 53), insufficient CT scan quality (n = 34) and indeterminate RT-PCR test results in case of normal CT scan (n = 17), leaving 144 eligible patients (Fig. 1).

Reference standard
As there has not been a definitive gold standard for the diagnosis of COVID-19 pneumonia so far, we chose to apply (high clinical suspicion for COVID-19 pneumonia) as reference standard for CT and RT-PCR test.o Cough, shortness of breath, difficulty of breathing, olfactory disorder, taste disorder, confusion, persistent pain or pressure in the chest, pale, gray, or blue-colored skin, lips, or nail beds, depending on skin tone, inability to awake or stay awake.o The patients were divided into two groups as follows: • Group 1: Low suspected group including patients with atypical signs and symptoms that were not compatible with category A or B, and had been under restricted isolation without contact with SARS-CoV-2 positive patients in the last 14 days and without travel history to an epidemic region.• Group 2: High suspected group including patients with symptoms compatible with either category A or B and with close contact with SARS-CoV-2 positive patients in the last 14 days or travel history to an epidemic region.
According to CDC, close contact to with SARS-CoV-2 positive patients is defined as being within 6 feet close to the patient for at least 15 minutes (cumulative over a 24-hour period).

CT
The CT scans were of patients with symptoms ≥ 5 days, to avoid misinterpretation in case of (normal) CT scan in early cases.The CT scans were evaluated according to the CO-RADS and CTSS.Two radiologists, blinded to the patients' symptoms and RT-PCR test, evaluated the CT scans independently.The radiologists had 13 and 30 years of experience, respectively.The participants were placed supine, arms above the head, and held breath at full inspiration.Initially, a topogram ensured covering from the base of the neck to the costophrenic angle.We used GE Revolution EVO CT scanner (GE, Milwaukee, USA), Auto mA: range 60-500 mA, 120 kV, noise-index 24.00, pitch 0.984:1, matrix 512, slice thickness 1.25 mm (3 mm sagittal and 5 mm coronal reconstructions) and beam collimation 40 mm.Window width level 1600/-600 Hounsfield units for the lungs and 450/55 Hounsfield units for the mediastinum.

The COVID-19 reporting and data system (CO-RADS)
Imaging analyses were according to the CO-RADS categorical system (Table 5).
In CO-RADS 1, the findings are normal or non-infectious, CO-RADS 2 is typical for other infections but not COVID-19 pneumonia, CO-RADS 3 resembles COVID-19 pneumonia but also other infections, CO-RADS 4 and 5 highly compatible with COVID-19 pneumonia but with atypical features as well in CO-RADS 4 (Table 5).

Statistical analysis
We used the SPSS version 25.0 (IBM Inc., Chicago, IL, USA) for statistical analysis.Continuous variables are presented as the median and range, and categorical variables as frequency and percentage.Differences between groups were analyzed using the Independent-Samples  were compatible with CO-RADS 3, 4 or 5 and moderate to severe CTSS, in the high suspected patients`group, the results were considered true positive (TP), and were considered false negative (FN) if they were compatible with CO-RADS 1 or 2 and mild CTSS.If the features of the CT scans were compatible with CO-RADS 1 or 2 and mild CTSS, in the low suspected patients`group, the results were considered true negative (TN), and were considered false positive (FP) if they were compatible with CO-RADS 3, 4, or 5 and moderate or severe CTSS.
The RT-PCR test sensitivity and specificity was calculated depending on the results of the initial test.If the first RT-PCR test was positive in the high suspected patiens` group, the result was considered TP, and FN if the test results were negative.If the initial RT-PCR test was negative in the low suspected patiens` group, the result was considered TN, and FP if the test results were positive.The Kappa method was used to quantify the inter-reader agreement.Kappa (κ value) was interpretated as follows: Kappa between 0.00 and 0.40 was considered as slight to fair agreement, Kappa between 0.41 and 0.80 moderate to substantial agreement and Kappa between 0.81 and 1.00 almost perfect agreement.

Statistical analysis of RT-PCR test findings
In total, 64 patients had positive initial RT-PCR tests and 80 patients had negative initial RT-PCR test, with a positive rate of 44.4% (95% CI 35%, 51%), accuracy (74.3%), positive predictive value (41.13%), negative predictive value (59.51%).The overall sensitivity and specificity of RT-PCR test in detecting COVID-19 pneumonia was (64.2%) and (93.9%), respectively (Table 7).Of the 64 patients with positive initial RT-PCR test results, 3 patients (2%)were categorized as low suspected group and their CT scans were compatible with CO-RADS 1 (N = 2) and CO-RADS 2 (N = 1) and had a mild CTSS.The remaining 61 patients with positive initial RT-PCR test were categorized as high suspected group and had CT findings compatible with CO-RADS 3 (N = 14), CO-RADS 4 (N = 19) and CO-RADS 5 (N = 28).There were 18 (13%) patients with the positive results came 1-2 weeks after the CT scan that showed changes compatible with COVID-19 pneumonia.The RT-PCR test of those patients was repeated 2-5 times as the initial results were negative.

Analysis of CT scans according to CO-RADS
All lobes were dominantly involved in CO-RADS 5 (Table 8).There was dominant right upper lobe (100%) and right lower lobe (91%) involvement in CO-RADS 4. In CO-RADS 3, lobe involvement predominated in the right upper lobe (83%), right lower lobe (89%), left upper Fig. 2 Axial unenhanced CT scan of the thorax of a 71-year-old male, with negative RT-PCR test for COVID-19 pneumonia.The patient presented with tirdness, loss of appetite, cough and dyspnea for 5 days.The CT scan was normal in accordance with CO-RADS 1 Fig. 3 A 75-year-old male with negative RT-PCR test for COVID-19 pneumonia.The patient presented with progressive cough and dyspnea over 7 days.An unenhanced CT scan showed small solid infiltrates (white arrows) and mild tree-in-bud infiltrates (black arrows) consistent with CO-RADS 2. The final diagnosis was infective bronchiolitis lobe (80%) and left lower lobe (83%).The left lower lobe was predominantly involved in CO-RADS 2 group (91%).
Other features included (Table 9): GGO in CO-RADS 4 and 5 (100% each), peripheral and subpleural distribution including the inter-septal fissures (91.3% CO-RADS 4 and 95% CO-RADS 5), vascular thickening (65.2% CO-RADS 4 and 81.1% CO-RADS 5), consolidations (61% CO-RADS 4 and 76% CO-RADS 5), crazy paving (52.2% CO-RADS 4 and 60% CO-RADS 5), pleural thickening (48% CO-RADS 4 and 73% CO-RADS 5) and subpleural bands (61% CO-RADS 4 and 73% CO-RADS 5).Lymphadenopathy was seen in 35% in CO-RADS 4 and 60% in CO-RADS 5. Pleural effusion was seen in 48% in CO-RADS 4 and 38% in CO-RADS 5. Mild GGO was predominant (100%), and subpleural involvement was common (82.2%) in CO-RADS 3 group, while the other features that were recognized in CO-RADS 4 and 5 were uncommon.Patchy opacities were present in 91%, and tree-in-bud opacities in 18.2% in CO-RADS 2, but both Fig. 4 A, B Axial unenhanced CT scan of the the lungs (A) and chest-radiograph, postero-anterior erect view (B) of a 67-year-old male with a positive RT-PCR test for COVID-19 pneumonia.The patient presented with sore throat, tirdness, cough and slight fever of 4 days duration.The CT scan that was performed on the 5th day after symptoms debut showed bilateral perihillar ground-glass opacities (black arrows) and semi solid patcy infiltrates (white arrows), which were compatible with CO-RADS 3. The infiltrates were subtle on the chest-radiograph taken on the 7th day after symptoms debut to ensure adequate drainage of the bilateral pleural exudate that was found on the CT scan patterns of opacities were not recognized in CO-RADS 4 and 5 (Table 9).Chest-radiograph failed to detect COVID-19 pneumonia in 17 (12%) patients with positive RT-PCR test (Fig. 4).Twelve patients had blank chest-radiographs performed within two days from the CT scan that showed CO-RADS 2 (n = 1), CO-RADS 3 (n = 7), CO-RADS 4 (n = 3), and crazy-paving pattern with CO-RADS 5 (n = 1).The chest-radiographs of three patients showed scattered nonspecific infiltrates.CT scans were performed on the same day in two patients and after four days in the 3rd patient, were compatible with CO-RADS 5. Pleura plaques have obscured the pneumonic infiltrates in one patient, who after two days underwent CT scan that showed CO-RADS 5.One patient with a poor health condition underwent a supine chest-radiographs that didn`t show signs of COVID-19 pneumonia.CT scan performed on the same day showed CO-RADS 3.

Table 7
The sensitivity and specificity of CT when categorized and analyzed according to CO-RADS and CTSS, respectively, compared to the sensitivity and specificity of RT-PCR test

Discussion
COVID-19 pneumonia is a highly contagious disease that led to global challenges to health care systems.Early diagnosis has been proven to be crucial for disease control and treatment [9,14].The definitive diagnosis of COVID-19 pneumonia occurs by means of positive RT-PCR test, yet it has limitations.RT-PCR test may take several days before it is obtainable and its availability has been challenging in the epidemic regions [3,4,7].Its efficiency depends on the adequate collection of the viral RNA, which varies between patients and even in a single individual during the course of the disease.Additionally, the efficiency of collecting the swabs varies according to the performer's experience, resulting in variable amounts of the collected material and increasing the chances of false-negative results [4].Due to its availability, CT has been embraced by numerous health centers as it has been helpful in alleviating the burden faced by the healthcare facilities [8,9].CO-RADS categorizes pulmonary changes in COVID-19 pneumonia according to the degree of suspicion based on CT findings, while CTSS estimates the degree of parenchymal lung involvement [12,14].The current study shows that CO-RADS has a direct positive relation with CTSS and they prevailed to depict cases of coronavirus disease when RT-PCR test and/or chest-radiograph failed.Correspondingly, studies by Lessmann et al. [11] and Leiveld et al. [12] showed that analyzing CT scans according to CO-RADS and CTSS has promoted the performance of CT in the diagnosis of COVID-19 pneumonia.We found that the specificity (93.9%) and the NPV (59.51%) of the initial RT-PCR test were superior to the specificity (76.8%) and NPV (45.18%) of CT, yet the sensitivity of CT (93.18%) was superior to that of the initial RT-PCR test (64.2%) in detecting COVID-19 pneumonia.In addition, PPV (55.41%), positive rate (66%) and accuracy (86.8%) of CT were higher in comparison with PPV (41.13%), positive rate (44.4%) and accuracy (74.3%) of RT-PCR test (Table 7).This is in accordance with a study by Fang et al. [15], that showed that CT has a higher sensitivity (98%) than the initial RT-PCR test (71%) in detecting COVID-19 Pneumonia.Ai et al. [7] showed similar results compared to ours, with 97% sensitivity of CT in detecting COVID-19 pneumonia and PPV of 65%.The same study showed lower specificity (25%) and accuracy (68%), respectively, compared to our results with specificity (76.8%) and accuracy (86.8%).This discrepancy can be due to differences in the studys' designs and in the included patients` populations.The current study showed a high inter-reader agreement with 88% κ value (95% CI 80%, 96%).In a study carried out by Prokop et al.
[8], the study population was divided into two groups: a group with positive RT-PCR test results as reference standard, and a group with negative RT-PCR test results using the high clinical suspicion as reference standard.After analyzing the two patient groups, the study showed a high inter-reader agreement in analyzing CT scan according to CO-RADS in the first group with positive RT-PCR test results (91% (CI 85%, 97%)) and even higher values were noticed in the second group with the high clinical suspicion as a reference standard (95% (CI 91%, 99%)), which was in accordance with our study.In our study, 18 patients (13%) had an initial negative RT-PCR test with CT scans suggestive of COVID-19 pneumonia.Ai T et al. [7] reported in their study 413 patients with negative initial RT-PCR test.Of those patients, 308 (75%) had CT features compatible with COVID-19 pneumonia.Likewise our study, two studies by Kortela et al. [3] and Clerici et al. [6], chose to refer to high clinical suspicion as reference standard, their studies showed that the sensitivity of the initial RT-PCR test in detecting COVID-19 pneumonia was [47.3%, 95% CI 44.4%, 50.3%)) and (77%, 95% CI (73%, 81%)), respectively, which was in accordance with our results.
Thirty-eight CT scans were compatible with CO-RADS 1, including 2 patients with positive RT-PCR test.Sixteen patients were discharged to home isolation, including the patients with positive RT-PCR test.The remaining patients had COPD exacerbation, malignant infiltrates, and granulomatous inflammation.They were admitted for treatment, accordingly.
Eleven CT scans showed tree-in-bud changes and/ or patchy opacities without GGO, which was atypical for COVID-19 pneumonia, consistent with CO-RADS 2. The one participant with a positive RT-PCT test was managed symptomatically.The remaining participants received treatment for bronchogenic pneumonia according to the causative agent.
GGO occurs in viral pneumonia in general [16].Their pathogenesis probably results from the incomplete filling of the air cavities with cells and liquids (such as edema, pus, and hemorrhage), interstitial thickening due to inflammation, edema, or fibrosis and partial alveolar depression, and alveolar damage, while the bronchopneumonia consolidations result from inflammatory reactions localized in patches around the bronchioles [17,18].
Most types of viral pneumonia share similar imaging features in the same Viridea family due to similarities in the pathogenesis [19,20], including Corona Viridea, thus the imaging features of COVID-19 pneumonia are comparable to those of other members of this viral family, like SARS-Co-V and MERS-Co-V.That can explain the indistinctness in CO-RADS 3 group, where the RT-PCR test was positive in 14 patients and negative in 21 patients.Both sub-categories had similar CT features, consisting of unsharp GGO or small nodular infiltrates with a background of mild GGO.
Chest-radiograph failed in detecting COVID-19 pneumonia in 17 (12%) patients.Twelve chest-radiographs were normal at the same time when CT revealed changes compatible with CO-RADS 2-4.The remaining chestradiographs were inconclusive as the pneumonic infiltrates were obscured by interstitial lunge changes or because of improper cooperation to the examination.Other studies found that the sensitivity of CT versus chest-radiograph was 97-98% versus 33-69%, respectively [28][29][30].
Our study has limitations.It was conducted in only two centers with a relatively small population.Only to observers evaluated the CT scans.We included only symptomatic participants, many of them had severe manifestations on CT.Asymptomatic participants weren`t included, which may have resulted in bias in selecting patients with a more manifest disease, hence affecting CO-RADS performance appraisal.

Conclusions
We found that RT-PCR test showed higher specificity and NPV compared to CT in detecting COVID-19 pneumonia, yet CT showed higher sensitivity, PPV, accuracy and positive rate when categorized and analyzed according to CO-RADS and CTSS compared to RT-PCR test.Although we didn't find it applicable solely to diagnose COVID-19 pneumonia, CT prevailed to show signs of COVID-19 pneumonia at early stages when RT-PCR test failed, thus we recommend to use it as a primary diagnostic tool, which can provide a significant advantage especially in the settings of rapid onset of a pandemic disease.
Kruskal-Wallis test and Pairwise Comparison test.Significance values were adjusted by the Bonferroni correction for multiple tests.A two-sided p value of less than 0.05 was considered statistically significant.Using high clinical suspicion as a reference standard, the sensitivity, specificity, positive predictive value (PPV), negative predicitve value (NPV) and accuracy of the chest CT scans and RT-PCR were calculated.The Wilson-score test was used to calculate the 95% confidence interval (CI) for the sensitivity.If the features of the CT scans

Fig. 1
Fig. 1 Flow diagram of the total study population showing the initial numbers of participants, exclusion criteria and the final number of the included eligible participants

Fig. 5 Fig. 6
Fig. 5 Axial chest CT scan of a 51-year-old female with positive RT-PCR test for COVID-19 pneumonia.The patient presented with cough, tiredness, dyspnea and fever of 2 days duration.Due to clinical suspicion of pulmonary embolism, a CT angiography of the thorax was performed on the 6th days after symptoms debut, showing unilateral and perihillar groud-glass opacities (white arrows) consistent with CO-RADS 4 Fig. 6 An unenhanced CT scan of the thorax of a 38-year-old male with positive RT-PCR test for COVID-19 pneumonia.The patient presented with dyspnea, severe cough, tiredness and loss of smell sense that persisted for at least 5 days.The scan showed bilateral severe ground-glass opacities (black stars) with reverse hallo sig in the right upper lobe consistent with CO-RADS 5 (black arrow)

CTSS:
CT severity score, NPV: negative predictive value, PPV: positive predictive value, RT-PCR test: Real-time reverse transcription-polymerase chain reaction *The patients were divided in two groups based on high or low clinical suspicion according to CDC COVID-19 case definition

Table 1
Characteristics of the study population

Table 2
Overview over the comorbidities of the study population

Table 3
The signs and symptoms of the current study population

Table 4
The paraclinical parameters of our patients

CO-RADS groups Level of suspicion Features
with or without consolidations, close to the pleura / the interlobar fissures.Rounded and unsharp demarcated GGO and sharply delineated GGO outlining the secondary pulmonary lobules, indicating early involvement of the lungs The "crazy paving" pattern appears later in the course of the disease, with visible interlobular lines and increasing consolidations within the GGO areas Opacities resembling organizing pneumonia, reverse halo sign and GGO with subpleural consolidations in the late course of the disease.Subpleural bands with thickened vessels

Table 8
Pulmonary lobar involvement correlated to COVID-19 pneumonia reporting and data system (CO-RADS) groups *T%/C%: The percentage within the individual CO-RADS category/ The percentage within all categories, CO-RADS: COVID-19 pneumonia reporting and data system

Table 9
Computed tomography features of coronavirus disease 2019 reporting and data system (CO-RADS) groups*T%/C%: The percentage within the individual CO-RADS category/ The percentage within all categories, CO-RADS: COVID-19 pneumonia reporting and data system