Multilayer Perceptron Predicting Cervical Lymph Node Metastasis for Papillary Thyroid Carcinoma

Abstract

Background: Lymph node metastasis is related to thyroid cancer recurrence; hence, early identification and prediction of cervical lymph node metastasis (CLNM) in thyroid cancer are essential.

Materials and methods: Ultrasound characteristics and patients’ clinical information for 478 thyroid nodules from 383 patients were collected, and a multilayer perceptron (MLP) was used to train and test the veracity to predict CLNM and form a network model. Sixty new patients with papillary thyroid carcinoma (PTC) were evaluated with the MLP neural network model. The metastasis status of these patients was then compared with the pathological results. The prediction of metastasis by the MLP and by multiple regression was compared.

Results: Calcification, age, sex, and maximum diameter were important predictive factors of CLNM by the MLP, and the area under the receiver operating characteristic curve was 0.715. No significant differences were found between the MLP and multiple regression in predicting CLNM. The average confidence of the model used in these new patients in predicting metastasis with PTC was 68.9%.

Conclusion: Ultrasound images from thyroid nodule characteristics and patients’ clinical information can be used as predictive factors of CLNM by the MLP method to a certain extent.

Main article text

Introduction

Recent data suggest that the increasing incidence of thyroid cancer may be starting to slow [1]; however, the incidence of thyroid cancer in several developed countries has markedly increased [2, 3]. In 2019, new cases of thyroid cancer account for 4% of all new cancers diagnosed in women [4], ranking sixth in the world. Although thyroid cancer has a good prognosis, the incidence of cervical lymph node metastasis (CLNM) can reach 30% to 80% [5, 6]. Studies have shown that lymph node metastasis is related to thyroid cancer recurrence [7, 8]; thus, early identification or prediction of CLNM in thyroid cancer is essential.

Artificial intelligence has been widely used in various fields and has quickly penetrated many aspects of modern life. In recent years, artificial intelligence has made rapid progress in the medical field [9], and there are a number of research studies have examined deep learning on massive data [10]. Artificial intelligence may provide an early diagnostic strategy for lymph node metastasis in patients with breast cancer with clinically negative lymph nodes [11]. Other aspects of medicine require more research based on artificial intelligence, for example, papillary thyroid carcinoma (PTC). Artificial intelligence has been used to optimize the Thyroid Imaging Reporting and Data System (TI-RADS) to validate the American College of Radiology TI-RADS and slightly improve specificity and maintain sensitivity [12]. A large number of data have verified the risk factors of CLNM in PTC, and the overall factors are consistent, including age, sex, maximum diameter, aspect ratio, and calcification [2, 13–15]. Although the use of artificial neural networks (ANNs) in predicting central lymph node metastases in papillary thyroid microcarcinoma had been researched, the prediction of lymph node metastases in PTC by a multilayer perceptron (MLP) has not been studied and needs further exploration [16]. At the same time, the MLP has many forms of charts that can show the important factors in predicting the outcome intuitively. This study aimed to explore the risk factors of lymph node metastasis according to preoperative thyroid ultrasound characteristics and clinically relevant patient information and to predict the potential of CLNM in PTC by using neural network MLP, thus guiding individualized clinical treatment.

Materials and methods

Patients

Information on patients with thyroid nodules who were treated at Shengjing Hospital of China Medical University and had pathology results from January 2018 to October 2019 was collected. A total of 1139 nodules from 896 patients were collected. Patients’ clinical information was collected through the HIS system, including information such as sex, age, family history, history of radiotherapy, and thyroid function tests. All patients underwent cervical ultrasound within 7 days prior to surgery, and the metastasis of both central and lateral cervical lymph nodes was recorded. Patients who had thyroidectomy or missing statistical data, those whose nodules were benign or who had other pathological types of thyroid cancer, and patients younger than 18 years were excluded. In total, 478 PTC nodules from 383 patients were collected in our statistical study, forming a neural network model ( Figure 1 ). After forming the model, the data for 60 new patients with PTC undergoing surgery from January to May 2020 in Shengjing Hospital of China Medical University were evaluated with the previously established model.

Figure 1

Patient selection.

Ultrasonic classification criteria

Ultrasonic testing of patients with PTC was performed before the operation by two ultrasound doctors with >5 years of experience. The ultrasound characteristics of thyroid nodules were recorded and analyzed in the following aspects: nodular echo (hypoechoic and equal/hyperechoic), size (≤1 and >1 cm), location (upper, middle, and lower poles), number (single and multiple), depth (close or not close to the envelope), tissue background (normal and Hashimoto’s thyroiditis), aspect ratio (>1 and ≤1), boundary (clear and unclear), shape (regular and irregular), calcification (calcification and non-calcification), blood distribution (detectable and undetectable), and posterior attenuation (with and without). According to the classification of the maximum diameter of nodules (≤1 or >1 cm), PTC with a maximum dimension <1 cm was differentiated as microcarcinoma [17, 18].

Pathological diagnosis

The lymph node status of each PTC in this study was obtained from the postsurgical pathological diagnosis report in our hospital. PTC with CLNM was defined as CLNM, and PTC without metastasis was defined as non-CLNM.

Statistical analysis

SPSS version 22.0 (IBM Corp., Armonk, NY, USA) was used in the statistical analysis. Logistic regression analysis was used to explore the different factors associated with CLNM, which was the criterion to predict the CLNM of PTC. A P value <0.05 was considered statistically significant.

The MLP, which is called an ANN, was used to train and test the veracity to predict the CLNM as a new method. The samples were split in the neural network by the default settings of SPSS. The architecture of the MLP model includes hidden and output layers; in the module of the hidden layers, the model had a hidden layer, and the activation function was chosen as the hyberbolic tangent and was automatically computed, whereas in the output layer, the activation function was identity. We formed a training set and a verification set according to the known data in a 7:3 ratio. The closer the areas under the receiver operating characteristic (ROC) curve are to 1, the higher the accuracy of the assessment. MedCalc software (MedCalc, Ostend, Belgium) was used to compare the metastasis status predicted by this neural network model with logistic regression. According to the MLP neural network model with .xml format constructed by the known data, the risk of metastasis of the 60 new patients with PTC was evaluated on the basis of the previously established model. Sex, age, maximum diameter, location, number, depth, tissue background, aspect ratio, boundary, shape, calcification, blood distribution, posterior attenuation, echo, family history, and thyroid function were used as factors evaluating lymph node metastasis. The metastasis status of these patients was then compared with the pathological results.

Results

Patient information and ultrasonic characteristics of thyroid nodules

The clinical information on 383 patients with 478 nodules of PTC were collected from our department, and the ultrasound characteristics of all the thyroid nodules were received before surgery. There were 309 (80.7%) female and 74 (19.3%) male patients. The age of the patients ranged from 22 to 80 years, and the average age was 45.8 years. A total of 214 (44.8%) nodules were CLNM, and 264 (55.2%) were non-CLNM ( Table 1 ).

Table 1

Characteristics of Patients’ Basic Information

Sex
Female	385 (80.5)
Male	93 (19.5)
Lymph node
Non-CLNM	264 (55.2)
CLNM	214 (44.8)
Family history
No	452 (94.6)
Yes	26 (5.4)
Thyroid function
Normal	259 (54.2)
Abnormal	219 (45.8)

Values are presented as frequency (percentage).

CLNM, cervical lymph node metastasis.

After integration of multiple factors, including the basic information on patients and the ultrasonic characteristics of thyroid nodules, the factors related to PTC were determined by multiple regression. Sex, age, thyroid function of the patients, maximum diameter, aspect ratio, and calcification of the thyroid nodules were the risk factors of CLNM ( Table 2 ). Male patients with PTC were 1.87 times more likely to have CLNM than women (P<0.05). Patients younger than 40 years were 3.21 times more likely to have CLNM than patients 40–60 years, who, in turn, were 2.72 times more likely to have CLNM than patients older than 60 years. Thyroid nodules with maximum diameter >1 cm had 1.86 times higher risk of CLNM than those with a diameter ≤1 cm. Patients with an aspect ratio ≤1 cm (odds ratio [OR]=1.58, P<0.05), calcification (OR=2.29, P<0.05), and abnormal thyroid function (OR=1.66, P<0.05) had a higher risk of CLNM than non-CLNM.

Table 2

Comparison of Ultrasonic Characteristics and Patients’ Clinical Information on Papillary Thyroid Cancer between the CLNM and non-CLNM groups

Metastasisa	B	Standard Error	Wald	Sig.	Exp(B)	95% Confidence Interval for exp(B)
						Lower Bound	Upper Bound
Intercept	0.180	0.866	0.043	0.836
Sex	–0.626	0.264	5.629	0.018*	0.535	0.319	0.897
Age
<40 years	1.166	0.278	17.641	<0.0001*	3.209	1.862	5.529
40–60 years	1.000	0.274	13.378	<0.0001*	2.720	1.591	4.649
Location
Upper pole	–0.328	0.271	1.467	0.226	0.720	0.423	1.225
Middle pole	–0.091	0.235	0.152	0.697	0.913	0.576	1.446
Number pole	–0.338	0.316	1.144	0.285	0.713	0.384	1.325
Depth (close to the envelope)	0.115	0.228	0.254	0.614	1.122	0.718	1.753
Tissue background (normal)	0.011	0.347	0.001	0.975	1.011	0.512	1.998
Maximum diameter	–0.620	0.231	7.171	0.007*	0.538	0.342	0.847
Aspect ratio	0.457	0.227	4.044	0.044*	1.579	1.012	2.464
Boundary	0.127	0.289	0.193	0.660	1.135	0.644	2.000
Shape	–0.025	0.231	0.012	0.913	0.975	0.620	1.533
Calcification	–0.830	0.230	13.021	<0.0001*	0.436	0.278	0.684
Blood distribution	0.258	0.222	1.349	0.245	1.294	0.838	2.000
Posterior attenuation	0.138	0.232	0.354	0.552	1.148	0.729	1.809
Echo	0.266	0.497	0.285	0.593	1.304	0.492	3.456
Family history	–0.312	0.427	0.532	0.466	0.732	0.317	1.691
Thyroid function tests	–0.510	0.214	5.652	0.017*	0.601	0.395	0.914

*P<0.05.

^aOccurred neck lymph node metastasis.

CLNM, cervical lymph node metastasis.

Factors related to lymph node metastasis by the MLP

Calcification, sex, age, and maximum diameter could be used to predict lymph node metastasis with MLP ( Figures 2 and 3 ). During the training stage, the percent correction of predicting non-CLNM was 75.4% compared with the pathology based on ultrasonic characteristics and clinical information. During the verification stage, the consistency predicting non-CLNM was 76.5% according to the training results ( Table 3 ). The area under the ROC curve for this method of predicting metastasis was 0.715 (P<0.001) ( Figure 4 ), thus indicating that it can meaningfully distinguish between lymph node metastasis and non-metastasis.

Figure 2

Importance of risk factors predicting lymph node metastasis based on the multilayer perceptron. Factors with >50% importance are calcification, sex, age, and maximum diameter.

Figure 3

Model. The first column of the model represents the input layer, which has 16 neurons, including sex, age, thyroid function result, maximum diameter, aspect ratio, calcification, echo, size, location, number, depth, tissue background, aspect ratio, boundary, shape, calcification, blood distribution, and posterior attenuation. The middle column is the hidden layer, with 9 neurons. The third column includes two output layers, which are cervical lymph node metastasis and non-metastatic.

Table 3

Lymph Node Metastasis Predicts Condition on the basis of the Training and Testing Sets by the Multilayer Perceptron

Sample	Observed	Predicted
		Non-CLNM	CLNM	Percent Correct
Training	Non-CLNM	138	45	75.4
	CLNM	76	81	51.6
	Overall percentage	62.9	37.1	64.4
Testing	Non-CLNM	62	19	76.5
	CLNM	25	32	56.1
	Overall percentage	63.0	37.0	68.1

Dependent variable: lymph node metastasis.

CLNM, cervical lymph node metastasis.

Figure 4

Area under the receiver operating characteristic curve (0.715) under the multilayer perceptron method predicting the cervical lymph node metastasis.

Comparison of the area under ROC curves on multiple regression and the MLP

The area under the ROC curve of multiple regression was 0.711, whereas that of MLP was 0.715 with MedCalc. The P value was 0.69 (Table 4), thus indicating that no significant differences were found between the MLP and multiple regression in predicting CLNM.

Table 4

Comparison of the Receiver Operating Characteristic Curves of Multiple Regression and the Multilayer Perceptron

Difference between areas	0.00361
Standard errora	0.00910
95% Confidence interval	–0.0142 to 0.0215
z Statistic	0.397
Significance level (P)	0.6916

^aDeLong et al.,1988.

Prediction of new patient metastasis with the known model

For 60 patients with PTC different from the previous patients, 36 cases were predicted to be consistent with the pathological results of the thyroid nodules, and 24 cases were different, according to the basic information of the patients and the ultrasound characteristics of the thyroid nodules. The average confidence of the model used in these new patients for predicting metastasis with PTC was 68.9%.

Discussion

Thyroid cancer has a higher proportion of CLNM [5]. However, the probability of preoperative routine ultrasound or computed tomography examination to detect metastatic lymph nodes is not sufficiently high [19, 20]. Therefore, we wanted to predict CLNM according to clinical information and the preoperative ultrasound characteristics of the thyroid nodules in order to provide a reference for surgery.

According to our data, patient sex, age, and thyroid function were significantly associated with CLNM by traditional multifactor analysis. We found that the patients with aspect ratio ≤1 cm (OR=1.59, P<0.05) more frequently had CLNM than non-CLNM. In a study [21] summarizing 1179 nodules, when the aspect ratio was ≤1, the nodules were mostly benign. In other words, an aspect ratio >1 may be a risk factor for malignant nodules, but among all malignant nodules, the number of nodules between the two groups with aspect ratios >1 and ≤1 was similar. Studies [22, 23] have also observed no significant relationship between aspect ratio and CLNM of thyroid cancer. A study [24] has shown that an aspect ratio >1 is a risk factor for central lymph node metastasis with Hashimoto’s thyroiditis. The conclusions are slightly different among different studies. This conclusion is related to the different backgrounds of thyroid nodules, to the limitation of thyroid papillary microcarcinoma, and to the limitation of central lymph nodes; thus, the conclusions of various documents differ. Thus, the influencing factors of PTC lymph node metastasis are not very powerful when the aspect ratio is >1, and thus require further study.

When the artificial intelligence MLP method was used to predict the veracity of CLNM, the clinical information and nodular ultrasound features of 478 PTC nodules were trained and tested. Seventy percent of the data were used for training to predict CLNM, and the remaining 30% were used to verify this prediction method. Cross contrasts the correct rate of the distinguished samples, showing that the probability of predicting non-CLNM was higher than that of CLNM, and the model’s ability to identify CLNM risk was lower than its ability to identify non-CLNM. Factors that affected the results by >50% were used as criteria for predicting lymph node metastasis, including calcification, age, sex, and maximum diameter.

Based on the above results, younger patients have a higher risk of metastasis. This may be related to body functions and metabolic conditions. Aging is the main risk factor for cancer progression, due to accumulation of mutations and a decline of homeostasis occurring during aging [25]. However, aging may not be a high-risk factor for metastasis. Paik et al. [26] believed that older patients have lower risk of recurrence of breast cancer, and the reason is not related to age itself but to the higher amount of estrogen-receptor protein in older patients’ breast tumors. A study has found a more progressive and higher recurrence rate of clinical PTC among young patients than older ones [27]. Therefore, a high probability of metastasis or recurrence of PTC at a young age may be related to hormone changes.

The maximum diameter and calcification factors of thyroid nodules were associated with CLNM in thyroid cancer in this study. The size of the largest diameter of thyroid nodules is related to CLNM. A study has shown that nodules >6 mm are a risk factor for metastasis in papillary thyroid microcarcinoma [28]. On the one hand, if the nodule is not found in the early stage and the treatment is not timely, the nodule may grow and have a high possibility of metastasis. On the other hand, some patients may choose regular follow-up instead of undergoing surgery. Suggestions should be given to patients and clinics to avoid thyroid lymph node metastasis when the nodules are >1 cm. Calcification is used as an American Thyroid Association guideline when evaluating thyroid nodules. Thyroid nodules with calcification have a higher risk of metastasis. It is believed that calcification is an independent risk factor for a large number of CLNM (number of metastatic lymph nodes >5) [29], and microcalcification is an indicator of rapid proliferation of cancer cells and may be a potential promoter of CLNM in PTC [30, 31].

SPSS adopts 0.5 as the probability boundary between right and wrong by default. In the independent variable importance map, the importance value was sorted in descending order. The proportion and importance of calcification, age, sex, and maximum diameter were seen when identifying CLNM. The area under the ROC curve was 0.715, thus indicating a predictive effect. Compared with binary logistic regression, the MLP method is clearer and more intuitive. The areas under the ROC curve between these two methods for predicting CLNM of thyroid cancer were 0.711 and 0.715, respectively. Both logistic regression and the MLP meet the needs for CLNM prediction. The areas under the curve were similar, but the MLP’s synaptic weights chart, lift chart, predicted versus observed chart, and other charts can more intuitively show the most important factors in predicting the outcome. The ANN method is not only more intuitive, but also more convenient and meaningful. By including the relevant ultrasound information of the patient’s nodule and clinical information, information can be obtained on whether the lymph node may metastasize. Currently, the model is not stable enough, and in further research, the sample size could be increased to improve its prediction veracity and stability.

According to the MLP neural network model, the risk of metastasis of 60 new patients with PTC was predicted, and the average confidence was 68.9%. To a certain extent, this model can be used to predict cervical lymph node metastases from PTC, but the accuracy of the prediction needs to be improved. In future studies, the data volume of the training model should be increased to make the predicting model more stable, and more new data should be collected for prospective research.

Conclusion

Ultrasonic images from thyroid nodule characteristics and patient clinical information can be used as factors to predict CLNM by the MLP. The important predictive factors are calcification, age, sex, and maximum diameter.

Acknowledgments

We are grateful to the staff of the Department of Epidemiology of China Medical University for help in the statistical analysis.

Conflict of interest

The authors declare no conflicts of interests.

Author contributions

Ying Huang designed the research; Jing-wen Shi collected the data and wrote the manuscript; Qi Zhang collected the data; Tian-tong Zhu collected the data.

Funding

This study was funded by 345 Talent Project, Health and Medical Big Data Research of China Medical University (item number HMB201902103), and Key Research and Development Guidance Plan of Liaoning Province (item number 2019JH8/10300008).

References

1. BR Roman, LG Morris, L Davies. The thyroid cancer epidemic, 2017 perspective. Curr Opin Endocrinol Diabetes Obes 2017;24:332–6. [PMID: 28692457 DOI: 10.1097/MED.0000000000000359]

2. S Vaccarella, S Franceschi, F Bray, CP Wild, M Plummer, et al. Worldwide thyroid-cancer epidemic? The increasing impact of overdiagnosis. N Engl J Med 2016;375:614–7. [PMID: 27532827 DOI: 10.1056/NEJMp1604412]

3. L Dal Maso, C Panato, S Franceschi, E Emoto, T Kasahara, et al. The impact of overdiagnosis on thyroid cancer epidemic in Italy, 1998-2012. Eur J Cancer 2018;94:6–15. [PMID: 29502036 DOI: 10.1016/j.ejca.2018.01.083]

4. RL Siegel, KD Miller, A Jemal. Cancer statistics, 2019. CA Cancer J Clin 2019;69:7–34. [PMID: 30620402 DOI: 10.3322/caac.21551]

5. KE Lee, IY Chung, E Kang, H Koo do, KH Kim, et al. Ipsilateral and contralateral central lymph node metastasis in papillary thyroid cancer: patterns and predictive factors of nodal metastasis. Head Neck 2013;35:672–6. [PMID: 22715063 DOI: 10.1002/hed.23016]

6. YG Eun, YC Lee, KH Kwon. Predictive factors of contralateral paratracheal lymph node metastasis in papillary thyroid cancer: prospective multicenter study. Otolaryngol Head Neck Surg 2014;150:210–5. [PMID: 24367047 DOI: 10.1177/0194599813514726]

7. YC Lee, SY Na, GC Park, JH Han, SW Kim, et al. Occult lymph node metastasis and risk of regional recurrence in papillary thyroid cancer after bilateral prophylactic central neck dissection: a multi-institutional study. Surgery 2017;161:465–71. [PMID: 27574773 DOI: 10.1016/j.surg.2016.07.031]

8. SK Kim, I Park, JW Woo, JH Lee, JH Choe, et al. Predictive factors for lymph node metastasis in papillary thyroid microcarcinoma. Ann Surg Oncol 2016;23:2866–73. [PMID: 27075321 DOI: 10.1245/s10434-016-5225-0]

9. P Hamet, J Tremblay. Artificial intelligence in medicine. Metabolism 2017;69S:S36–40. [PMID: 28126242 DOI: 10.1016/j.metabol.2017.01.011]

10. LQ Zhou, JY Wang, SY Yu, GG Wu, Q Wei, et al. Artificial intelligence in medical imaging of the liver. World J Gastroenterol 2019;25:672–82. [PMID: 30783371 DOI: 10.3748/wjg.v25.i6.672]

11. LQ Zhou, XL Wu, SY Huang, GG Wu, HR Ye, et al. Lymph node metastasis prediction from primary breast cancer US images using deep learning. Radiology 2020;294:19–28. [PMID: 31746687 DOI: 10.1148/radiol.2019190372]

12. B Wildman-Tobriner, M Buda, JK Hoang, WD Middleton, D Thayer, et al. Using artificial intelligence to revise ACR TI-RADS risk stratification of thyroid nodules: diagnostic accuracy and utility. Radiology 2019;292:112–9. [PMID: 31112088 DOI: 10.1148/radiol.2019182128]

13. C Liu, C Xiao, J Chen, X Li, Z Feng, et al. Risk factor analysis for predicting cervical lymph node metastasis in papillary thyroid carcinoma: a study of 966 patients. BMC Cancer 2019;19:622. [PMID: 31238891 DOI: 10.1186/s12885-019-5835-6]

14. KS Zheng, Y Zeng, C Chen, YZ Wu, B Chen, et al. [Risk factors of cervical lymph node metastasis in papillary thyroid microcarcinoma: an analysis based on data from the Surveillance, Epidemiology and End Results database]. Zhongguo Yi Xue Ke Xue Yuan Xue Bao 2018;40:736–43. [PMID: 30606382 DOI: 10.3881/j.issn.1000-503X.10747]

15. K Back, JS Kim, JH Kim, JH Choe. Superior located papillary thyroid microcarcinoma is a risk factor for lateral lymph node metastasis. Ann Surg Oncol 2019;26:3992–4001. [PMID: 31264121 DOI: 10.1245/s10434-019-07587-2]

16. S Ozden, S Er, B Saylam, BD Yildiz, K Senol, et al. A comparison of logistic regression and artificial neural networks in predicting central lymph node metastases in papillary thyroid microcarcinoma. Ann Ital Chir 2018;89:193–8. [PMID: 30588918]

17. AL Mitchell, A Gandhi, D Scott-Coombes, P Perros. Management of thyroid cancer: United Kingdom National Multidisciplinary Guidelines. J Laryngol Otol 2016;130:S150–60. [PMID: 27841128 DOI: 10.1017/S0022215116000578]

18. K Huang, Z Bai, D Bian, P Yang, X Li, et al. Diagnostic accuracy of contrast-enhanced ultrasonography in papillary thyroid microcarcinoma stratified by size. Ultrasound Med Biol 2020;46:269–74. [PMID: 31703968 DOI: 10.1016/j.ultrasmedbio.2019.10.001]

19. E Kim, JS Park, KR Son, JH Kim, SJ Jeon, et al. Preoperative diagnosis of cervical metastatic lymph nodes in papillary thyroid carcinoma: comparison of ultrasound, computed tomography, and combined ultrasound with computed tomography. Thyroid 2008;18:411–8. [PMID: 18358074 DOI: 10.1089/thy.2007.0269]

20. JL Roh, JY Park, JM Kim, CJ Song. Use of preoperative ultrasonography as guidance for neck dissection in patients with papillary thyroid carcinoma. J Surg Oncol 2009;99:28–31. [PMID: 18942072 DOI: 10.1002/jso.21164]

21. FS Ouyang, BL Guo, LZ Ouyang, ZW Liu, SJ Lin, et al. Comparison between linear and nonlinear machine-learning algorithms for the classification of thyroid nodules. Eur J Radiol 2019;113:251–7. [PMID: 30927956 DOI: 10.1016/j.ejrad.2019.02.029]

22. K Sun, X He, L Guo. [Clinical and ultrasonic features of papillary thyroid microcarcinoma and risk factors for central lymph node metastasis]. Lin Chung Er Bi Yan Hou Tou Jing Wai Ke Za Zhi 2021;35:260–3. [PMID: 33794614 DOI: 10.13201/j.issn.2096-7993.2021.03.015]

23. F Li, D Pan, Y He, Y Wu, J Peng, et al. Using ultrasound features and radiomics analysis to predict lymph node metastasis in patients with thyroid cancer. BMC Surg 2020;20:315. [PMID: 33276765 DOI: 10.1186/s12893-020-00974-7]

24. W Zhao, L He, J Zhu, A Su. A nomogram model based on the preoperative clinical characteristics of papillary thyroid carcinoma with Hashimoto’s thyroiditis to predict central lymph node metastasis. Clin Endocrinol (Oxf) 2021;94:310–21. [PMID: 32984984 DOI: 10.1111/cen.14302]

25. L Raffaghello, V Longo. Metabolic alterations at the crossroad of aging and oncogenesis. Int Rev Cell Mol Biol 2017;332:1–42. [PMID: 28526131 DOI: 10.1016/bs.ircmb.2017.01.003]

26. F Li, D Pan, Y Wu, J Peng, Q Li, et al. A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med 2004;351:2817–26. [PMID: 15591335 DOI: 10.1056/NEJMoa041588]

27. Y Ito, A Miyauchi, M Kihara, T Higashiyama, K Kobayashi, et al. Patient age is significantly related to the progression of papillary microcarcinoma of the thyroid under observation. Thyroid 2014;24:27–34. [PMID: 24001104 DOI: 10.1089/thy.2013.0367]

28. X Zheng, C Peng, M Gao, J Zhi, X Hou, et al. Risk factors for cervical lymph node metastasis in papillary thyroid microcarcinoma: a study of 1,587 patients. Cancer Biol Med 2019;16:121–30. [PMID: 31119052 DOI: 10.20892/j.issn.2095-3941.2018.0125]

29. XP Huang, TT Ye, L Zhang, RF Liu, XJ Lai, et al. Sonographic features of papillary thyroid microcarcinoma predicting high-volume central neck lymph node metastasis. Surg Oncol 2018;27:172–6. [PMID: 29937168 DOI: 10.1016/j.suronc.2018.03.004]

30. N Wang, Y Xu, C Ge, R Guo, K Guo. Association of sonographically detected calcification with thyroid carcinoma. Head Neck 2006;28:1077–83. [PMID: 17022090 DOI: 10.1002/hed.20481]

31. Q Wu, YM Zhang, S Sun, JJ Li, J Wu, et al. Clinical and sonographic assessment of cervical lymph node metastasis in papillary thyroid carcinoma. J Huazhong Univ Sci Technolog Med Sci 2016;36:823–7. [PMID: 27924514 DOI: 10.1007/s11596-016-1669-5]