- Open Access
Comparison of the RSNA chest CT classification system and CO-RADS system in reporting COVID-19 pneumonia in symptomatic and asymptomatic patients
Egyptian Journal of Radiology and Nuclear Medicine volume 53, Article number: 122 (2022)
Coronavirus disease (COVID-19) is a new infection with three pandemic waves up till now. CT plays an important role in diagnosis with multiple reporting systems that can be used during CT analysis. We aimed to compare reporting using the recommendations of the radiological society of North America (RSNA) versus the coronavirus disease reporting and data system (CO-RADS) and to assess the performance of CT if used in asymptomatic patients as a screening. Two hundred and fifty-one patients who underwent chest CT scanning either due to clinical suspicion or as screening before hospital admission were included in this retrospective observational cross-sectional study. This was followed by RT-PCR for confirmation. Three radiologists with different years of experience interpreted the CT findings using the RSNA recommendations and the CO-RADS reporting. The data were collected and compared.
There was no statistically significant difference noted in the diagnostic accuracy obtained while using the RSNA recommendations and the CO-RADS reporting system. Also, a good inter-rater agreement was noticed while using the two reporting systems. The CT showed a highly significant value while used in the assessment of symptomatic patients in controversy to the screening of asymptomatic patients.
Both reporting systems show similar diagnostic accuracy with a good almost similar inter-rater agreement. Both can be used while interpreting the CT images of cases with suspected COVID-19 infection. CT can be used effectively in the detection of COVID-19 infection between symptomatic patients while it is of a lower value in the screening of asymptomatic patients.
Coronavirus disease 2019 (COVID-19) infection was first reported in December 2019 in Wuhan, China . The widespread human-to-human transmission led to the declaration of a pandemic by the World Health Organization (WHO) on January 30, 2020 .
Although the gold standard for diagnosis of COVID-19 is the reverse transcription-polymerase chain reaction (RT-PCR) assay, its sensitivity ranges from 42 to 83% depending on the viral load, in addition, there is a relative scarcity in some developing countries that occurred during the first wave and the long time it takes for the results to be delivered . With the COVID-19 pandemic, many rapid tests appeared depending on rapid detection of the IgG and IgM within 10 to 30 min providing a rapid decision yet still of much lower diagnostic accuracy compared to the RT-PCR test .
This led to the evolution of the role of computed tomography (CT) in many countries as a screening tool for patients with clinically suspected COVID-19 pneumonia . COVID-19 has CT-specific features yet may show some overlap with other diseases especially viral pneumonia 
In an attempt to standardize CT reporting in patients with suspected COVID-19, the Radiological Society of North America (RSNA) Expert Consensus Statement was published . In early March 2020, the Dutch Radiological Society developed a standardized reporting system for reporting suspected cases of COVID-19 pneumonia, which was named COVID-19 Reporting and Data System (CO-RADS) . Reporting cases requires the presence of high interobserver agreement to avoid confusion and guide physicians during patient management . This was the hypothesis of this article trying to show the degree of interobserver agreement between different reporting systems that are commonly used during CT interpretation of cases with suspected COVID-19 infection to avoid clinicians’ confusion while using any of them. To our knowledge, limited studies are available comparing the two reporting systems and correlating them to RT-PCR findings, and showing the difference in CT accuracy while used in the screening of asymptomatic patients.
We aimed to compare the current reporting systems for suspected cases of COVID-19 pneumonia (RSNA reporting system and CO-RADS classification) and to correlate them with RT-PCR results. Also, we assessed the degree of interobserver agreement in both reporting systems.
The secondary aim was to assess the performance of CT in the screening of asymptomatic patients.
Two hundred and fifty-one patients were referred from the Emergency Room (ER) or the Triage clinic of Ain Shams University Hospital (a tertiary hospital that was allocated to partially receive COVID-19 cases) to the Radiology Department for a non-contrast CT study of the chest to exclude the presence of COVID-19 infection, followed by RT-PCR that had been taken within the same day. They were enrolled in this retrospective observational cross-sectional study. The study was conducted over the period from June 2020–May 2021. The authors received the ethical approval to use the patients’ data and images in the Picture Archiving and Communication System (PACS) from the Faculty of Medicine - Ain Shams Unniversity Ethical Committee, and the patients’ consent was waived being a retrospective study.
Any patient with clinical suspicion of COVID-19 infection, and any asymptomatic patients who were indicated for hospital admission or prepared for any surgical interference who underwent non-contrast CT to screen for COVID-19 infection, followed by RT-PCR.
Patients with non-available RT-PCR results and patients with excessive breathing motion artifacts that affected the accuracy of the CT image interpretation.
The patients’ records were revised to collect their demographic data including age and sex. Also, the presence of any comorbidities was recorded (Diabetes mellitus DM, hypertension HTN, cardiac disease, chronic kidney disease, chronic liver disease, and chronic respiratory disease). The patients presenting symptoms were recorded from their records to classify them into cases having symptoms of COVID-19 infection and asymptomatic cases. Symptomatic cases were defined as those with any of the following symptoms (Fever, cough, dyspnea, sore throat, malaise, myalgia, anosmia, ageusia, and gastrointestinal symptoms including anorexia, nausea, and diarrhea,) 
The study was performed using the Prime Aquilion 80-slice CT machine (Toshiba, US). The patients were placed in a supine position with their arms elevated above their heads. All patients were instructed to hold their breaths with full inspiration for as long as they could, to minimize the breathing motion artifacts.
CT parameters included tube voltage of 120 kV, mA ranging between 150 and 400 mA according to the patient’s weight, 1.25 mm slice thickness, 0.625 mm slice interval, 512 × 512 matrix, and tube speed 35 mm/rotation (0.5 s rotation time). In pediatric cases, a low -dose CT technique was used via reducing the Kv and mA, and in some pediatric cases with the motion not controlled, oral sedation was used under the supervision of an anesthesiologist.
Image processing and interpretation: The images were transferred to the workstation (Fuji Synapse workstation or Paxera-Ultima workstation) for image interpretation by three independent readers. (One radiologist with 14 years of experience, the second with 10 years of experience, while the third was a fellow with only 5 years experience in chest radiology.) All radiologists depend on their observation and experience without using any special algorithms or deep learning systems to measure the inter-rater agreement. Each reader was blinded to the results of RT-PCR, the clinical condition of the patients apart from being a suspicious case of COVID-19 infection, and the interpretation of the other radiologists.
The following radiological points were recorded to estimate the COVID-19 infection probability:
Presence of ground-glass opacity (GGO) with or without septal thickening, crazy paving pattern, consolidation, and fibrosis.
The distribution of the GGO within the lung was either peripheral or central, unilateral or bilateral, and uni-lobar or multi-lobar.
The presence of a reverse halo sign.
The presence of other CT features of pneumonia as lobar consolidation, pulmonary cavity/abscess, and pulmonary nodules with a tree in bud appearance.
Each CT scan was evaluated according to the RSNA recommendations and CO-RADS reporting system as follows:
Reporting the case according to the RSNA recommendations into the typical, indeterminate, atypical, or negative probability for COVID-19 infection .
Reporting the case according to CO-RADS into CO-RADS 1–5 with CO-RADS 1 was considered negative, while CO-RADS 5 was considered a very high probability case .
The presence of bilateral multi-lobar predominantly peripheral patchy areas of GGO was considered the most typical form of COVID-19 infection consistent with CO-RADS 5.
Nasopharyngeal or oropharyngeal swab was done for each patient on the same day of the CT study. The RT-PCR tests were repeated twice only for the cases with negative swabs and persistent clinical suspicion.
Sample size calculation
Using PASS program version 11 for sample size calculation (NCSS, LSS Statistical Software, USA), setting power at 80% and alpha error at 0.05, according to the previous literature the expected AUC for diagnostic validity of CO-RADS for diagnosis of COVID-19 pneumonia is about 0.80, assuming a difference of 0.10 in AUC between CO-RADS and RSNA reporting system for diagnosis of COVID-19 pneumonia, a sample size of at least 150 suspected patients were needed (75 cases with positive RT-PCR and 75 cases with negative RT-PCR).
Data were analyzed using IBM© SPSS© Statistics version 23 (IBM© Corp., Armonk, NY). Continuous numerical variables are presented as mean and SD and categorical variables as numbers and percentages or ratios.
Inter-rater agreement was examined using the intraclass correlation coefficient (ICC). We used a two-way random model for a single measurement to calculate absolute agreement. The ICC is interpreted as follows: ICC values less than 0.5 are indicative of poor agreement, values between 0.5 and 0.75 indicate moderate agreement, values between 0.75 and 0.9 indicate good agreement, and values greater than 0.90 indicate excellent agreement according to Koo and Li 
Diagnostic accuracy for RSNA and CO-RADS classification was calculated for each Rater using RT-PCR as the gold standard for diagnosis. The following diagnostic indices were calculated: sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and correct classification and misclassification rates.
This was a retrospective study conducted on 251 patients who underwent non-contrast CT and RT-PCR. One hundred and thirty-five patients presented with symptoms of suspicion for COVID-19 infection, while the other 116 patients came for hospital admission or preoperative and performed the RT-PCR as well as the CT as screening to avoid the spread of infection inside the non-isolated areas inside the hospital.
The mean age of the patients was 52 ± 19 years ranging from 5 to 98 years. One hundred and thirty-seven patients were males representing 54.6%. Regarding the patients’ comorbidities, cardiovascular diseases were the most common (157/251) (62.5%) followed by DM (79/251) (31.5%) (Fig. 1).
The number of patients with positive RT-PCR for COVID-19 was 136 (54.2%) patients. The proportion of patients with positive RT-PCR for COVID-19 in typical, indeterminate, atypical, and negative categories according to the RSNA classification system ranged from 73.6–90.4%, 46.7–50.9%, 22.9–34%, and 38.2–48%, respectively. The proportion of patients with positive RT-PCR for COVID-19 in CORADS 1, 2, 3, 4, and 5 categories ranged from 40–48%, 25.6–30.6%, 42.9–45.2%, 50–79.2%, and 73.6–91.1%, respectively (Table 1).
Regarding the RSNA reporting system
The sensitivity and specificity of the CT using the typical criteria established by RSNA recommendations ranged from 47–49% to 80–94%, respectively, for the three raters. The sensitivity and specificity of the typical and indeterminate categories together ranged from 65–68% to 52–71%, respectively, for the three raters. The positive predictive value while using the typical criteria ranged from 81 to 90%, while the negative predictive value ranged from 58 to 61%. The positive predictive value for typical and indeterminate groups ranged from 63 to 73%, while the negative predictive value ranged from 58 to 64%. (Table 2). Reporting using the RSNA reporting system showed good inter-rater agreement with ICC = 0.77 (Table 3) (Figs. 2, 3, 4, 5).
Regarding the CO-RADS reporting system
The sensitivity and specificity of the CO-RADS 5 category ranged from 38–49% to 80–96% respectively for the three raters. The sensitivity and specificity of CO-RADS 4 and 5 categories together ranged from 51–62% to 66–91% respectively for the three raters. The sensitivity and specificity of CO-RADS 3, 4, and 5 categories together ranged from 65–68% to 52–71% respectively for the three raters. The positive predictive value for CO-RADS 5 group ranged from 74 to 91%, while the negative predictive value ranged from 56 to 57%. The positive predictive value for CO-RADS 4 and 5 groups ranged from 72 to 88%, while the negative predictive value ranged from 58 to 61%. The positive predictive value for CO-RADS 3, 4 and 5 groups ranged from 63 to 73%, while the negative predictive value ranged from 58 to 64%. (Table 4). Reporting using the CO-RADS reporting system showed good inter-rater agreement with ICC = 0.75 which was close and similar to the RSNA reporting system (Table 5) (Figs. 2, 3, 4, 5).
Regarding the use of CT in screening asymptomatic patients
For simplicity and with the base of good inter-rater agreement found between the three raters in the current study, we used the reading of rater 1 who is the more experienced in the analysis of the results among symptomatic and asymptomatic cases. Among 43 asymptomatic cases with positive RT-PCR test for COVID-19 infection, the CT detected changes of high to indeterminate probability for COVID-19 infection in only 17 cases (39.5%). While in 93 symptomatic cases with positive RT-PCR, the CT detected changes of high to indeterminate probability for COVID-19 infection in 72 cases (77.4%). The CT showed a highly significant P-value of 0.000 when used among clinically suspected patients in controversy to a non-significant P-value of 0.95/0.96 when used as screening in asymptomatic patients (Table 6). Negative CT and indeterminate probability were the highest interpretation detected among asymptomatic patients during the use of the RSNA reporting system, while in the CO-RADS reporting system, CO-RADS 1 and 3 were the commonest.
In this study, the authors tried to evaluate the inter-observer agreement while reporting the CT using RSNA and CO-RADS reporting systems and showed good inter-observer agreement even with the difference in the radiologists’ experiences. Also, almost similar diagnostic accuracy was found while using the two reporting systems. CT showed non-significant values in the detection of COVID-19 infection while used in the screening of asymptomatic patients.
The American College of Radiology and the Royal College of Radiologists recommended that CT should not be used to screen patients with suspected COVID-19 infection [11, 12]. However, chest CT was used in our hospital as a method of triaging patients with suspected COVID-19 infection, as well as to exclude infection in patients considered for hospitalization for other reasons, especially with the scarcity and time-consuming nature of the RT-PCR test and if there was any clinical suspicion of COVID-19 infection . The RSNA chest CT classification for reporting COVID 19 pneumonia provides a method for conveying to the clinicians the level of suspicion of COVID-19 . In March 2020, the Dutch Radiological Society developed the CO-RADS reporting system to classify pulmonary involvement by COVID-19 .
In this study, we found the proportion of patients with RT-PCR confirmed COVID-19 was highest in the typical followed by the indeterminate category of the RSNA system. The proportion of RT-PCR confirmed cases in the negative category (38.2–48%) was higher than that in the atypical category (22.9–34%). This is concurrent with De Jaegere et al.  who explained this by the higher prevalence of other lung diseases in the atypical category (e.g., bacterial pneumonia and pulmonary edema) compared to the negative category, which was also the case in our study. Regarding the CO-RADS system, similar proportions were reported for RT-PCR confirmed cases with the CO-RADS 1 category showing a higher proportion of positive cases (40–48%) than the CO-RADS 2 category (25.6–30.6%). This was also in agreement with De Jaegere et al. 
Reporting using the RSNA reporting system showed good inter-rater agreement with ICC = 0.77. This is slightly lower than Byrne et al.  who reported substantial agreement in the indeterminate category and almost perfect agreement in the other three categories. De Jaegere et al.  reported substantial to a moderate overall agreement. An Italian study by Ciccarese et al.  reported a moderate interobserver agreement.
We found that reporting using the CO-RADS reporting system showed good inter-rater agreement with ICC = 0.75 which is slightly higher than Prokop et al.  who reported moderate reliability with a k value of 0.47. Fujioka et al.  reported substantial to an almost perfect interobserver agreement in the CO-RADS system with ICC 0.800–0.874 which is considered slightly higher than that in our study.
The sensitivity of the typical category in the RSNA reporting system ranged from 47 to 49% for the three readers. When using the indeterminate category as the threshold, the sensitivity for typical and indeterminate categories increased to 65–68%. The sensitivity in our study is remarkably less than that reported by Som et al.  who reported a sensitivity of typical and indeterminate findings to be 97.5% (range 94–100%). This could likely be explained by the use of CT in our hospital for symptomatic patients suspected to be infected by COVID-19 as well as a screening tool for patients considered for hospitalization for other reasons, while in their study, CT was used as a problem-solving tool in suspected cases and to guide management, which implied that all patients were symptomatic at the time of imaging, which was not the case in our study. Our findings concur with Kim et al.  who reported that CT may be normal in asymptomatic and early cases of COVID-19 infection. A meta-analysis done by Kwee et al.  reported a pooled sensitivity of 65.2% for the typical category and 90.2% for typical and indeterminate categories, also higher than the values in the current study.
The sensitivity of the CO-RADS 5 category ranged from 38 to 49%. When using CO-RADS 4 and 5 categories as a threshold, the sensitivity ranged from 51 to 62%. The sensitivity of CO-RADS 3, 4, and 5 categories together increased to 65–68% in our study. Our values were lower than those by Kwee et al.  who reported a pooled sensitivity of CO-RADS-5 category 70.4%, for CO-RADS 4 and 5 categories increased to 85.8%, and for CO-RADS 3, 4, and 5 categories were reaching 92.5%.
Regarding the specificity of the typical category in our study, it ranged from 80–94% and decreased to 52–71% for the typical and indeterminate categories, concurrent with Som et al.  who reported a specificity of 54.7% for the latter categories. The high specificity of the typical category was higher than that reported in a large study in China by Ai et al. , who reported a specificity of 25% for chest CT. This was explained in their study by the possibility of RT-PCR being falsely negative in many cases. In these patients, they performed a further analysis based on exposure history, clinical manifestations, and serial CT scans, to better classify patients with positive CT findings and negative RT-PCR tests. Similarly, we reported the specificity of CO-RADS 5 category to range from 80 to 96%, CO-RADS 4 and 5 ranged from 66 to 91% and decreased to 52–71% for CO-RADS 3, 4 and 5 categories, concurrent with the literature stating that at high diagnostic thresholds, specificity increases at the cost of sensitivity 
Finally, our results concerning the non-significant value of CT while used in the screening of asymptomatic patients are consistent with the ACR recommendations [11, 12]. Smet et al.  found that CO-RADS 3 was the most commonest among positive asymptomatic cases and concluded CT screening for asymptomatic patients is not recommended which is similar to the current study.
Our study had several limitations. First the use of RT-PCR as a gold standard for COVID-19 infection with its reported false-negative rate that could reach 63% in nasopharyngeal swabs . Second, RT-PCR was not repeated in all negative cases, yet in patients with persistent clinical suspicion of COVID-19, at least two RT-PCR tests were done. The study was also limited by its retrospective design and the lack of clinical data regarding the duration of symptoms at the time of CT scanning. Among the strengths of the study was the inclusion of symptomatic and asymptomatic cases rendering it more valuable in the assessment of the diagnostic performance of CT without selection bias.
There is a good inter-rater agreement while reporting using either the RSNA recommendations or the CO-RADS reporting systems among radiologists with different levels of experience, rendering them useful in providing standardized reports to convey the level of suspicion of COVID-19 infection to the clinicians. Despite the high specificity of CT in the diagnosis of symptomatic COVID-19, especially with high diagnostic thresholds, our study concluded that its sensitivity is lacking owing to the considerable number of confirmed COVID-19 cases in the negative and atypical categories, limiting its use as a screening test in asymptomatic cases in controversy to what happened while used in symptomatic patients. So, it is not suggested to use CT as screening for COVID-19 infection in asymptomatic patients.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Corona Virus Disease 2019
Reverse transcription-polymerase chain reaction
Radiological society of North America
Coronavirus disease reporting and data system
Zhu N, Zhang D, Wang W et al (2020) A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 382(8):727–733. https://doi.org/10.1056/NEJMoa2001017
Mahase E (2020) China coronavirus: WHO declares international emergency as death toll exceeds 200. BMJ 368:m408
Li Y, Yao L, Li J et al (2020) Stability issues of RT-PCR testing of SARS-CoV-2 for hospitalized patients clinically diagnosed with COVID-19. J Med Virol 92(7):903–908. https://doi.org/10.1002/jmv.25786
Böger B, Fachi MM, Vilhena RO et al (2021) Systematic review with meta-analysis of the accuracy of diagnostic tests for COVID-19. Am J Infect Control 49(1):21–29. https://doi.org/10.1016/j.ajic.2020.07.011
Prokop M, Everdingen W, Vellinga T et al (2020) CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19—definition and evaluation. Radiology 296(2):E97–E104. https://doi.org/10.1148/radiol.2020201473
Bai HX, Hsieh B, Xiong Z et al (2020) Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology 296(2):E46. https://doi.org/10.1148/radiol.2020200823
Simpson S, Kay FU, Abbara S et al (2020) Radiological society of North America expertconsensus statement on reporting chest CT findings related to COVID-19. endorsed by the society of thoracic radiology, the American college of radiology, and RSNA. Radiol Cardiothorac Imaging 2(2):e200152. https://doi.org/10.1148/ryct.2020200152
De Jaegere T, KrdzalicJ FB et al (2020) Radiological society of North America chest CT classification system for reporting COVID-19 pneumonia: interobserver variability and correlation with RT-PCR. Radiol Cardiothorac Imaging 2(3):e200213. https://doi.org/10.1148/ryct.2020200213
Gandhi RT, Lynch JB, Rio C (2020) Mild or moderate Covid-19. N Engl J Med 383:1757–1766
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012
ACR Recommendations for the use of chest radiography and computed tomography (CT) for suspected COVID-19 infection. https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID-19-Infection. Accessed 31 Mar 2020
RCR position on the role of CT in patients suspected with COVID-19 infection. Accessed 31 Mar 2020. The role of CT in patients suspected with COVID-19 infection | The royal college of radiologists (rcr.ac.uk).
Rubin G, Ryerson C, Haramati L et al (2020) The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner society. Radiology 296:172–180. https://doi.org/10.1148/radiol.2020201365
Byrne D, O’Neill SB, Muller NL et al (2020) RSNA expert consensus statement on reporting chest CT findings related to COVID-19: interobserver between chest radiologists. Can Asssoc Radiol J 72:159–166. https://doi.org/10.1177/0846537120938328
Ciccarese F, Coppola F, Spinelli D et al (2020) Diagnostic accuracy of North America expert consensus statement on reporting CT findings in patients with suspected COVID-19 infection: an Italian single-center experience. Radiol Cardiothorac Imaging 2(4):e200312. https://doi.org/10.1148/ryct.2020200312
Fujoka T, Takahashi M, Mori M et al (2020) Evaluation of the usefulness of CO-RADS for chest CT in patients suspected of having COVID-19. Diagnostics (Basel) 10(9):608. https://doi.org/10.3390/diagnostics10090608
Som A, Lang M, Yeung T et al (2020) Implementation of the radiological society of North America expert consensus guidelines on reporting chest CT findings related to COVID-19: a multireader performance study. Radiol Cardiothorac Imaging 2(5):e200276. https://doi.org/10.1148/ryct.2020200276
Kim D, Quinn J, Pinsky B et al (2020) Rates of co-infection between SARS-Cov-2 and other respiratory pathogens. JAMA 323(20):2085–2086. https://doi.org/10.1001/jama.2020.6266
Kwee RM, Adams HJA, Kwee TC (2021) Diagnostic performance of CO-RADS and the RSNA classification system in evaluating COVID-19 at chest CT: a meta-analysis. Radiol Cardiothorac Imaging 3(1):e200510. https://doi.org/10.1148/ryct.2021200510
Ai T, Yang Z, Hou H et al (2020) Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology 296(2):E32. https://doi.org/10.1148/radiol.2020200642
Smet KD, Smet DD, Ryckaert, et al (2021) Diagnostic performance of chest CT for SARS-CoV2 infection in individuals with or without COVID-19 symptoms. Radiology 298(1):E30–E37
Zitek T (2020) The appropriate use of testing for COVID-19. West J Emerg Med 21(3):470–472. http://www.ncbi.nlm. nih.gov/pubmed/32302278.
No funding was obtained for this study.
Ethics approval and consent to participate
These were taken from our institute ethical committee (Faculty of Medicine-Ain shams university) with written informed consents waived being a retrospective study.
Consent for publication
The written informed consents were waived being a retrospective study.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sheha, A.S., Mohamed, N.H., Eid, Y.M. et al. Comparison of the RSNA chest CT classification system and CO-RADS system in reporting COVID-19 pneumonia in symptomatic and asymptomatic patients. Egypt J Radiol Nucl Med 53, 122 (2022). https://doi.org/10.1186/s43055-022-00798-w
- Computed tomography (CT)