Diagnostic accuracy, reliability, and reviewer agreement of a new proposed risk prediction model for metastatic cervical lymph node from head and neck squamous cell carcinoma using MDCT

Abstract

Background

Nine-point risk scoring system for metastatic cervical lymph nodes has been developed to be incorporated into clinical practice for further management and better prognosis for head and neck squamous cell carcinoma (HNSCC). It is based on suspicious computed tomography (CT) scanning findings. This study aimed to assess the risk scoring system validity and reliability for diagnosing cervical lymph node metastasis from head and neck squamous cell carcinomas.

Results

The intra-class correlation (ICC) was utilized to assess the inter-observer agreement. We had 102 malignant lymph nodes and 60 benign lymph nodes based on histopathological results. Based on a lymph node (LN) based analysis regarding the LNs categorized as scores 4 to 9 for diagnosing metastatic cervical lymph nodes, the risk scoring system had a sensitivity, specificity, and an accuracy of 89.2 to 91.2%, 68.3 to 70%, and 82.1 to 83.3%, respectively, depending on the observer. The inter-reviewer agreement (IRA) for the total score was excellent (ICC = 0.936). The optimal cutoff value for diagnosing metastatic cervical lymph nodes was > score 3.

Conclusions

Based on imaging findings, a risk scoring system for diagnosing metastatic cervical lymph nodes from head and neck squamous cell carcinoma was validated. This risk scoring system is a valuable guide for better decision-making.

Background

Risk factors for head and neck squamous cell carcinoma (HNSCC) comprise exposure to tobacco, alcoholism, and infection with an oncogenic virus [1]. Metastatic lymph node substantially influences HNSCC prognosis and treatment, and the ideal imaging modality for neck staging and lymph node metastasis diagnosis is contrast-enhanced multidetector CT (CE-MDCT) [2, 3]. As predictors of lymph node (LN) metastasis, a simple nine-point scoring model has been developed utilizing preoperative CT lymph node characteristics (comprising presence of necrosis or cystic change, diameters, and the ratio of long to short-axis diameter) in conjunction with primary tumor characteristics (such as T-stage) [3].

Nodes greater than 10 mm in their short axes are deemed abnormal. Even so, 20% of nodes larger than 10 mm lack metastatic deposits and show only hyperplasia histologically. In contrast, 23% of nodes with extracapsular spread measure less than 10 mm. Metastatic involvement is indicated by the presence of nodal necrosis, regardless of size. Even though this sign is specific to LN metastasis, it has limited importance in clinical practice [4].

On the basis of those CT features, this proper diagnostic model maximizes the role of CE- MDCT in metastatic cervical LN prediction and decision-making in daily practice as the presence and extent of metastatic lymph nodes affect lymph node dissection extent and therapeutic plans in HNSCC [5].

The risk scoring system that predicts metastatic cervical lymph node with newly diagnosed HNSCC patients consists of T stage of primary tumor T1, or T2 takes 0 points, and T3 or T4 takes 1 point. LN with shortest axial diameter < 1 cm takes 0-point, 1–2 cm takes 1 point, and ≥ 2 cm takes 4 points. Long to short axis (L/S) ratio ≥ 1.5 takes 0 points, and < 1.5 takes 1 point. The presence of necrosis takes 3 points, and its absence takes 0 point [6].

Risk categorization is estimated as follows: 0–1: low risk (17% of metastases), 2–4: intermediate risk (17% to 78% of metastases), and 5–9: high risk (≥ 78% of metastases) [6].

This work aimed to evaluate the reliability and validity of this proposed 9-point scoring system to detect and diagnose cervical lymph node metastasis in HNSCC patients’ using CE-MDCT suspicious parameters.

Methods

We followed the Standards for Reporting Diagnostic Accuracy (STARD) statement guidelines when conducting this diagnostic test accuracy study [7]. The institutional review board of Zagazig University has approved this study (ZU-IRB approval number: 9774).

Study design and population

This single-center prospective study enrolled 65 patients ranging in age from 27 to 80 years. Between November 2020 and November 2021, they were transferred from the oncology department to the radiology department. Before the study, the local institutional review board (IRB) and informed consent from all participants were obtained. We followed the ethical principles of the Declaration of Helsinki.

Inclusion criteria were (1) patients with pathologically confirmed head and neck cancer, especially squamous cell carcinoma (SCC), (2) those who had a neck CT scan before head and neck malignancy treatment, (3) patients with suspicious LNs for malignancy by the US. Exclusion criteria were (1) patients with serum creatinine level above 2 mg/dl, (2) patients with contrast media allergy (n = 1), (3) patients refused signing a consent despite informed discussion with the radiologist (n = 2), (4) pregnant or lactating females, (5) patients underwent radiation therapy and/or chemotherapy prior to LN dissection (6) suboptimal CT images (eg, motion artifacts and no coronal reformation) (n = 2) and (7) indeterminate histopathological results (n = 2).

The patients who met our inclusion criteria were 65 patients (28 females and 37 males), with a mean age of 53.88 ± 12.92 years. Patients' basic data are reported in Table 1. The study flowchart is demonstrated in Fig. 1.

Table 1 Patients’ basic characteristics of the studied groups

Fig. 1

The flowchart of the studied population shows excluded and included participants, frequency of the scoring system by each observer, and histopathological findings

Once registered, participants underwent comprehensive history taking and a detailed clinical assessment (General and Local). Laboratory investigations, including renal function tests. MDCT imaging using intravenous contrast media and finally histopathological analysis.

Protocol of CE-MDCT

Patient preparation

No special preparation was needed, only fasting for 1 h before the examination. Sedation was not required prior to examination.

Scan protocol and parameters

Philips Medical System, 128™ ingenuity CT, Nederland was utilized for the CT scan, with the following parameters: Tube voltage 120–140 kV, tube current 100–300 mA and detector collimation 64 × 0.75 mm, the pitch of one, the rotation period of 0.5 s, and reconstructed slice width of 3 mm and increment of 3 mm. The patient was lying on the table in a supine position with his arms beside his body. A scanogram covering the area from the frontal sinus upper margin to the aortic arch was acquired. The CT scans were acquired 60 s after administration of a bolus of iopromide (Ultravist 370, Bayer HealthCare) to ensure complete opacification of the neck arterial & venous systems. The dosage was administered into an antecubital vein at a flow rate of 3 ml/s. The iopromide dosage was estimated relying on the patient's body weight (1.5 ml/kg). The image acquisition was performed in a single breath-hold (from 10 to 15 s).

Image analysis

A sophisticated workstation received all of the CT scans (Philips Medical System, Best, The Netherlands). Two independent, experienced observers with 5 years-experience reviewed and interpreted all CT scans. Both observers independently assessed suspicious characteristics of lymph node metastasis on preoperative CT scans for each LN, as follows: (1) Diameter of LN (longest and shortest axial diameter), (2) the (L/S) ratio, (3) presence or absence of necrosis, (4) lymph node agglomeration, (5) nearby adjacent soft tissue infiltration, (6) laterality to the side of the primary tumor.

The longest diameter was determined and defined as the largest LN diameters in the axial and coronal planes. The shortest axial diameter was determined and defined as the largest diameter perpendicular to the longest axial diameter. Both observers selected slices for measuring LN diameter individually. The longest maximal axial diameter/shortest maximal axial diameter was utilized for calculating the L/S ratio. Necrosis was defined as low central density with irregular or rim-like enhancement of the remaining lymphatic tissue using the visual examination. Adjacent soft-tissue infiltration was defined as ill-defined lymph node margins or fat stranding in the neck. Each observer reviewed the CT-based parameters of the risk scoring system individually and assigned a score for each lymph node. The raters were blinded to patients’ names and any imaging interpretation. They were directed to the findings through the number of the series/image, site of primary tumor, level of LN and some spatial identifying information due to presence of multiple LNs.

Reference standard

LNs were definitively diagnosed based on histopathological results following surgery or Ultrasound (US)-guided biopsy.

US-guided biopsy was done either using a fine needle in an FNA or core needle varies and US transducer, which may have a needle guide. Multiple short passes through the lesion were performed. Multiple samples were obtained in a session.

Comparison between histopathological results and CT findings was done on a node-by-node basis by pairing the side, level and size of the detected LN by CT. Surgeons were provided by detailed data about the side, level, and gross features of LN in concern and the specimen was labeled and numbered paired with CT.

Sample size and power calculation

Chung et al. [6], who developed this 9-point scoring system, reported sensitivity and specificity of 74% and 95.3%. Assuming a prevalence rate of 38% (similar to their estimate), the present study will require at least a total of 59 lymph nodes (with at least 24 of them being positive) to have 80% statistical power to detect similar sensitivity and specificity in the population. The sample size was calculated using the Statistics and Sample Size app for Android (version 14) according to the steps outlined by Negida et al. [8]. Two snapshots from the sample size calculation software inputs and outputs are shown in Additional file 1.

Statistical analysis

Statistical Package for Social Science (SPSS) version 26 was utilized to code, input, present, and analyze the obtained data. Frequencies and percentages were utilized for qualitative data representation. Mean ± standard deviation (SD) (for normally distributed data) and median with interquartile range for quantitative variables (for not normally distributed data). When the P value was < 0.05 and < 0.001, the results were deemed statistically significant and highly statistically significant, respectively. Absolute agreement and reliability between methods were assessed by calculating "intraclass correlation coefficient r" to evaluate the relationship between calculated scoring systems and their subclasses by the two observers. The ICC values were analyzed as follows: poor agreement = 0.01–0.20; fair agreement = 0.21–0.40; moderate agreement = 0.41–0.60; good agreement = 0.61–0.80; and excellent agreement = 0.81–1.0. To measure validity, predictive value for positive (PVP), predictive value for negative (PVN), specificity, sensitivity, and accuracy at the 95% CI were calculated. For calculating the cutoff value and area under the curve (AUC) for predicting the metastatic cervical LNs, the receiver operating characteristic (ROC) curve was utilized.

Results

One hundred and sixty-two lymph nodes from 65 HNSCC patients were included in the study. CE-MDCT examinations and histopathological evaluations were done successfully with no side effects. Based on histopathological results, we had 102 malignant lymph nodes and 60 benign lymph nodes. The prevalence rate for malignant lymph nodes was 62.96%. The most common sites of primary tumors were the right buccal region and right side of the tongue, which were seen in 18.46%.

The 9-point scoring system parameters and total score for each lymph node

The frequency distribution of CT suspicious parameters of the 9-point scoring system and the total score of each lymph node by two observers are detailed in Tables 2 and 3. 56.2% of masses were reported to be stage T1 or T2 by the two observers. Most of the masses (86.4% reported by observer 1 and 89.5% reported by observer 2) had the shortest diameter of < 2 cm, and more than half of the masses had an L/S ratio < 1.5 cm. Observer 1 reported that (46.9%) of masses had areas of necrosis versus (42.6%) reported by observer 2. The scores 3 and 7 (14.2%) were the most common scores reported by observer 1. The score 6 (14.8%) was the most common score reported by observer 2.

Table 2 Frequency distribution of the different items of the scoring system by the 2 observers

Table 3 Frequency distribution of the scoring system by the 2 observers

The diagnostic performance in diagnosing metastatic cervical LN

The diagnostic performance of the risk scoring system for diagnosing cervical LN metastasis using a LN-based analysis is illustrated in Table 4. According to the observers, the accuracy, specificity, and sensitivity of the 9-point scoring system were 82.1 to 83.3%, 68.3 to 70%, and 89.2 to 91.2%, respectively.

Table 4 Validity of the scoring system versus histopathology as a gold standard

Inter-reviewer agreement for findings of the CE-MDCT and the total scoring system

Inter-reviewer agreement (IRA) for CE-MDCT based parameters and the total score results are presented in Table 5. All the subscales of the scoring system are highly correlated. T stage and shortest diameter subscales were the most correlated between the scoring system items (ICC = 0.950–0.960, respectively). The overall agreement for the total score was excellent (ICC = 0.936).

Table 5 Intraclass correlation coefficient between the 2 observers of different items and total score of the scoring system

ROC analyses

According to the observers, we analyzed the risk scoring system diagnostic performance data for the identification of the optimal cutoff value to diagnose cervical LN metastasis utilizing the ROC curve (Fig. 2). According to ROC analyses, both observers concurred that the cutoff value for prediction of metastatic cervical LN was > score 3. The application of this cutoff value was related to AUC ranges from 0.887 to 0.878, sensitivity from 89.2 to 91.2% (95% CI), and specificity from 68.3 to 70% (95% CI), depending on the observers.

Fig. 2

ROC curve analyses of both observers for the diagnostic performance of the scoring system for metastatic cervical LN detection

Some representative cases of the current study are illustrated in Figs. 3, 4, and 5.

Fig. 3

A 54-year-old male SCC patient suffered from pain, ulceration, and swelling. Contrast-enhanced MDCT neck a axial view shows bilateral glottis, supra glottis, aryepiglottic folds soft tissue mass measures 30 × 36 × 20 mm (white arrow). b Axial view shows left side LN enlargement at level (III) measures 9 × 13 mm with necrosis (score 4) (sky blue arrow). c The axial view shows bilateral LN enlargements at level (II, III and IV) (score 0) (orange arrows). Histopathology confirmed the node to be malignant at level III (sky blue arrow); others were benign

Fig. 4

A 46-year-old female patient with right side floor of the mouth SCC suffered from RT mandibular swelling. Contrast-enhanced MDCT neck a axial view shows ill-defined infiltrating soft tissue mass intimately related to the right mandibular body measures about 40 × 15 mm with submental extension and eroding the mandibular body on both sides (white arrow), and LT side LN enlargement at level II measures 7 × 12 mm (score 1) (sky blue arrow). b Axial view shows RT side LN enlargement at level III measures 42 × 32 mm with necrosis (score 9) (orange arrow). Histopathology confirmed the node to be malignant at level III and benign at level II

Fig. 5

A 64-year-old male with lower lip SCC patient suffered from painful bleeding red and white patch on the lip. Contrast-enhanced MDCT neck a axial view shows lower lip faintly enhanced soft tissue mass measures 54 × 49 mm (white arrow). b Axial view shows LT side LN enlargement at level IA measures 10 × 10 mm (score 3) (orange arrow). c Axial view shows RT side LN enlargement at level IB measures 5 × 7 mm (score 2) (sky blue arrow) and RT side LN enlargement at level IIA measures 5 × 7 mm (score 2) (orange arrow). Histopathology confirmed the node to be malignant at level IA and benign at level IB & IIA

Discussion

There were several attempts to develop risk scoring systems for diagnosing the possibility of lymph node metastases in different organs such as the breast, endometrium, esophagus, stomach, skin, and thyroid [9,10,11,12,13,14,15]. Given the poor prognosis for metastatic lymph nodes patients, the risk scoring system might be a useful tool not only for patients seeking counseling but also for clinicians during treatment planning [16].

Chung et al. developed a simple nine-point risk scoring system for metastatic cervical lymph nodes from head and neck squamous cell carcinoma based on CT imaging data with high prediction performance [6].

This 9-point scoring system contributes to decreased interobserver variability by providing objective evidence for diagnosis [6].

An attempt was performed to evaluate the reliability and diagnostic performance of a newly proposed predictive model for metastatic cervical LNs in HNSCC. This risk scoring system can facilitate patient management by physicians and improve communication between radiologists and physicians. Therefore, this 9-point scoring system can be applied conveniently in clinical practice.

A few studies have explained in detail and clarified the limits of this risk scoring system. However, since the Chung et al. [6] in 2019 has developed this new 9-point scoring system, no study has been done to assess its external validity or reliability, but other studies have developed other new risk scoring systems to stratify metastatic cervical lymph nodes from papillary thyroid carcinoma and HPV related oropharyngeal squamous cell carcinomas relying on suspicious imaging features of lymph node metastasis. To the best of our knowledge, this is the first study to validate this new nine-point scoring system by Chung et al. [6] for the detection of malignant metastatic cervical lymph nodes.

The overall findings are encouraging, demonstrating that this 9-point scoring system has a good diagnostic performance in the prediction of patients with cervical LN metastases. According to the observers, the risk scoring system performed well in diagnostic validity terms, with sensitivity (89.2 to 91.2%), specificity (68.3 to 70%), and accuracy (83.3 to 82.1%). These results are unsurprising considering that they are dependent on CT findings that have been extensively validated in several studies [17, 18] and have been demonstrated to be a reliable modality for metastatic cervical LN diagnosis. However, in our study, the high prevalence of metastatic LNs (62.9%) might be a potential selection bias affecting specificity and sensitivity calculations. Furthermore, the high prevalence of metastatic LNs in our study might explain the relatively high positive predictive values (83.2–83.5%). It should not escape our knowledge that our study has a high prevalence of cervical LN metastasis, and this high prevalence might be explained by the fact that we conducted the study in a major central hospital with medical and surgical oncology clinics that treat patients with head and neck cancers in the region. The high sensitivities were attributable to various reasons: First, we utilized a customized Philips workstation with updated facilities. Second, we precluded patients whose pathological findings were indeterminate. Third, we precluded patients whose images were of suboptimal quality (motion artifact). Fourth, two highly experienced radiologists assessed all images. This newly proposed 9-point scoring system application is easy and simple. The 9-point scoring system evaluated in our study is still not commonly used by oncologists and surgeons. Therefore, the details of this risk scoring system must be explained to the referring oncologists in multiple scientific meetings before the beginning of the study. Further, without evidence on the reliability and validity of this new 9-point scoring system, its significance for clinical practice would be limited. Thus, interobserver agreement was evaluated for the different CT parameters of the scoring system and for the total score. The overall findings were quite highly satisfactory. This scoring system showed excellent IRA (ICC = 0.936). In terms of its CT-derived parameters, T stage, shortest diameter, L/S ratio, and necrosis also showed an excellent IRA (ICC = 0.950, 0.960, 0.852, and 0.866, respectively).

This is the first study that assessed the interobserver agreement of this new risk scoring system. However, as regards the L/S ratio and necrosis features, the IRA was less than other features (ICC = 0.852 and 0.866, respectively). In terms of the L/S ratio, our findings were consistent with recent literature [19], which indicates that the location of a lesion may affect the measurement accuracy. Across all readers in the same study, both intra- and interobserver agreement were highest for pulmonary lesions, followed by hepatic lesions, and then lymph nodes. As lesion size increases, intra- and interobserver variability in measurement decreases. Our study showed a relatively high prevalence of small-sized LNs, and this could explain IRA for L/S ratio. As the proper assessment of the size of small lesions on CT is still challenging, this parameter may need further modification. In terms of necrosis, our IRA could be explained by the high number of small LNs, which hinder necrosis evaluation in some situations. Partial volume artifact of normal fatty hila may mimic nodal necrosis as well [20]. A small focus of necrosis may be missed during the assessment, so we suggest a further classification of necrosis into no necrosis, focal necrosis, and gross or cystic necrosis as described in the new reporting system of LN (Node RADS) [21]. Nodal necrosis in the presence of HNSCC is the most valuable sign of metastatic involvement, with specificity between 95 and 100% [20].

Although histopathology is the gold standard for diagnosing metastatic malignant LN, surgery is not risk-free. Most of the complications are mild, but serious complications might develop during surgery. So, the most convenient method for cervical LN metastatic prediction may be the noninvasive diagnosis. Integrating the new proposed 9-point scoring system into clinical practice may help to reduce much-referral for surgery and encourage further appropriate follow-up care for patients undergoing a diagnostic workup of cervical LN metastasis. This is crucial in patients with intermediate scores.

In this study, both reviewers strongly agreed that the optimal cutoff value of the risk scoring system for cervical LN metastasis prediction was > score 3 through the utilization of the ROC curve. This cutoff value was associated with the sensitivity ranges from 89.2 to 91.2%, according to observers. Our cutoff was in line with Chung et al. [6], who reported a score of > 3 as the optimal cutoff for cervical LN metastasis. Additionally, we suggest additional studies with a larger population to confirm or disprove this cutoff value. Finally, consistent with previous study results and based on our findings, the new proposed 9-point scoring system is considered a risk scoring system of CT suspicious parameters for cervical LN metastasis with great and several advantages, especially if it could be coupled with some management recommendations. Therefore, we strongly encourage the incorporation of this risk scoring system into CT reports.

Nevertheless, the 9-point scoring system has limitations as some important data are precluded from the scoring system. (e.g., site and level of LNs, number of LNs, LN margin, presence of calcification, extracapsular spread, and infiltration of surrounding structures). Therefore, Additional modification is required for the 9-point scoring system to be accurate and comprehensive of all essential definitions. For illustrating the added value of these data to the risk scoring system, large longitudinal studies on prognosis are needed.

The study had some limitations. First, the study was done in a single center. Hence, in future studies, we recommend affirmation through large multicenter studies. Second, in our study, the high prevalence of malignant metastatic cervical LNs may be a potential selection bias that might influence specificity and sensitivity calculations. Third, CT scans were assessed by highly qualified observers, which may affect diagnostic performance. Therefore, more studies are required about this risk scoring system performance when performed by inexperienced observers. Fourth, there was no follow-up of benign LNs because including a follow-up period carries the possibility of classifying the new malignant transformation over time as positive which might lead to biased evaluation of the accuracy parameters. Fifth, PET-CT was not involved in the clinical workup in this study. Although PET-CT has been more reliable than CT or MRI in the identification of metastasis, PET-CT spatial resolution may limit small (5 mm) intranodal metastatic deposits identification [22]. Finally, this new proposed 9-point scoring system may still need some modifications. So, we recommend the following diagnostic algorithm for LNs scanning/scoring in HNSCC patients (Fig. 6).

Fig. 6

The diagnostic algorithm for scoring cervical lymph nodes from head and neck squamous cell carcinoma

Conclusions

This new proposed 9-point scoring system for metastatic cervical LNs in HNSCC patients is very diagnostic and reproducible, and it is quite useful for oncologists in clinical decision-making.