Abstract
Purpose
In this study, we developed radiomic models that utilize a combination of imaging features and clinical variables to distinguish endometrial cancer (EC) from normal endometrium on routine computed tomography (CT).
Methods
A total of 926 patients consisting of 416 endometrial cancer (EC) and 510 normal endometrium were included. The CT images of these patients were segmented manually, and divided into training, validation, testing and external testing sets. Non-texture and texture features of these images with endometrium or uterus as region of interest were extracted. The clinical feature “age” was also included in the feature set. Feature selection and machine learning classifier were applied to normalized feature set. This manual optimized combination was then compared with the best pipeline exported by Tree-Based Pipeline Optimization Tool (TPOT) on testing and external testing set. The performances of these machine learning pipelines were compared to that of radiologists.
Results
The manual expert optimized pipeline using the “reliefF” feature selection method and “Bagging” classifier on the external testing set achieved a test ROC AUC of 0.73, accuracy of 0.73 (95% CI 0.62–0.82), sensitivity of 0.64 (95% CI 0.45–0.79), and specificity of 0.78 (95% CI 0.65–0.87), while TPOT achieved a test ROC AUC of 0.79, accuracy of 0.80 (95% CI 0.70–0.87), sensitivity of 0.61 (95% CI 0.43–0.77), and specificity of 0.90 (95% CI 0.78–0.96). When compared to average radiologist performance, the TPOT achieved higher test accuracy (0.80 vs. 0.49, p < 0.001) and specificity (0.90 vs. 0.51, p < 0.001), with comparable sensitivity (0.61 vs. 0.46, p = 0.130).
Conclusion
Our results demonstrate that automatic machine learning can distinguish EC from normal endometrium on routine CT imaging with higher accuracy and specificity than radiologists.
Similar content being viewed by others
Code availability
This code is available at https://github.com/HurongCSU/Endoproj.
Data availability
The images and clinical data are not available for public use to protect patient information.
References
Siegel R.L., Miller K.D., Jemal A (2017) Cancer Statistics. CA Cancer J Clin. 67(1), 7-30 (2017).
Moore K., Brewer M.A. Endometrial Cancer: Is This a New Disease?, Am Soc Clin Oncol Educ Book. 37, 435–442.
Lee YC, Lheureux S, Oza AM (2017) Treatment strategies for endometrial cancer: current practice and perspective, Curr Opin Obstet Gynecol. 29(1), 47-58.
Barwick T.D, Rockall AG, Barton DP, Sohaib SA (2006) Imaging of endometrial adenocarcinoma. Clinical Radiology, 61(7), 545-555.
Coussoou C, Laigle-Quérat V, Loussouarn D, Vaucel E, Frampas E (2020) Magnetic Resonance Imaging for local preoperative staging in endometrial cancer: Nantes local experience, Gynecol Obstet Fertil Senol.
Hardesty L.A, Sumkin JH, Hakim C, Johns C, Nath M (2001) The Ability of Helical CT to Preoperatively Stage Endometrial Carcinoma, American Journal of Roentgenology. 176(3), 603-606.
Braun M.M., Overbeek-Wager E.A., Grumbo R.J (2016) Diagnosis and Management of Endometrial Cancer, Am Fam Physician. 93(6), 468-74.
Rizzo S, et al (2018). Radiomics: the facts and the challenges of image analysis, European Radiology Experimental. 2(1), 36.
Gillies R.J., Kinahan P.E., Hricak H (2016) Radiomics: Images Are More than Pictures, They Are Data, Radiology. 278(2), 563-577.
Parmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL (2015) Machine Learning methods for Quantitative Radiomic Biomarkers, Sci Rep. 5, 13087.
Tongtong Liu, et al (2017) A mRMRMSRC feature selection method for radiomics approach, Conf Proc IEEE Eng Med Biol Soc. 616–619.
Mayerhoefer ME, et al (2020) Introduction to Radiomics, J Nucl Med.
Sidey-Gibbons, J.A.M., Sidey-Gibbons C.J (2019) Machine learning in medicine: a practical introduction, BMC Medical Research Methodology. 19(1), 64.
Günakan E, et al (2019) A novel prediction method for lymph node involvement in endometrial cancer: machine learning, Int J Gynecol Cancer. 29(2), 320-324.
Olson R.S., et al (2016) Automating Biomedical Data Science Through Tree-Based Pipeline Optimization, Lecture Notes in Computer Science. vol 9597.
Le T.T., Fu W, Moore J.H (2020) Scaling tree-based automated machine learning to biomedical big data with a feature set selector, Bioinformatics. 36(1), 250-256.
Su X, et al (2019) Automated Machine Learning Based on Radiomics Features Predicts H3 K27M Mutation in Midline Gliomas of the Brain, Neuro Oncol.
Orlenko A, et al (2019) Model selection for metabolomics: predicting diagnosis of coronary artery disease using automated machine learning (AutoML), Bioinformatics.
Andriy Fedorov, et al (2012) 3D Slicer as an image computing platform for the Quantitative Imaging Network, Magnetic Resonance Imaging. 30(9), 1323-1341.
Zwanenburg A, L.S., Vallières M, Löck S (2016) Image biomarker standardisation initiative, arXiv preprint. arXiv:161207003.
Zwanenburg A, et al (2020) The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping, Radiology. 10, 191145.
Jundong Li, et al (2018) Feature selection: A data perspective, ACM Computing Surveys (CSUR). 50(6), 94.
Pedregosa, et al (2011) Scikit-learn: Machine Learning in Python. JMLR. 12, 2825-2830.
Alan Agresti, Brent A. Coull (1998) Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions, The American Statistician. 52(2), 119-126.
Vallières, M., et al (2017). Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer, Scientific Reports. 7(1), 10117.
Vallières, M, Freeman CR, Skamene SR, El Naqa I (2015) A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities, Phys Med Biol. 60(14), 5471-96.
Bi Q, et al (2019). The Diagnostic Value of MRI for Preoperative Staging in Patients with Endometrial Cancer: A Meta-Analysis, Acad Radiol.
Alcazar J.L, et al (2015) Transvaginal ultrasound for preoperative assessment of myometrial invasion in patients with endometrial cancer: a systematic review and meta-analysis. Ultrasound Obstet Gynecol. 46(4), 405-13.
Alcázar JL, et al (2019) Diagnostic Performance of Transvaginal Ultrasound for Detecting Cervical Invasion In Women With Endometrial Carcinoma: A Systematic Review and Meta-analysis, J Ultrasound Med. 38(1), 179-189.
Bollineni VR, Ytre-Hauge S, Bollineni-Balabay O, Salvesen HB, Haldorsen IS (2016) High Diagnostic Value of 18F-FDG PET/CT in Endometrial Cancer: Systematic Review and Meta-Analysis of the Literature, J Nucl Med. 57(6), 879-85.
Deng L, et al (2015) The Combination of Diffusion- and T2-Weighted Imaging in Predicting Deep Myometrial Invasion of Endometrial Cancer: A Systematic Review and Meta-Analysis, J Comput Assist Tomogr. 39(5), 661-73.
Alcázar JL, et al (2017) Transvaginal ultrasound versus magnetic resonance imaging for preoperative assessment of myometrial infiltration in patients with endometrial cancer: a systematic review and meta-analysis, J Gynecol Oncol. 28(6), e86.
Luomaranta A, Leminen A, Loukovaara M (2015) Magnetic resonance imaging in the assessment of high-risk features of endometrial carcinoma: a meta-analysis, Int J Gynecol Cancer. 25(5), 837-42.
Xu X, et al (2019) Multiplanar MRI-Based Predictive Model for Preoperative Assessment of Lymph Node Metastasis in Endometrial Cancer, Front Oncol. 9, 1007.
Xie H, et al (2019) Preliminary utilization of radiomics in differentiating uterine sarcoma from atypical leiomyoma: Comparison on diagnostic efficacy of MRI features and radiomic features, Eur J Radiol. 115, 39-45.
De Bernardi E, et al (2018) Radiomics of the primary tumour as a tool to improve 18F-FDG-PET sensitivity in detecting nodal metastases in endometrial cancer, EJNMMI Res. 8(1), 86.
Neacşu A, et al (2018) Clinical and morphological correlations in early diagnosis of endometrial cancer, Rom J Morphol Embryol. 59(2), 527-531.
Colombo N, et al (2016) ESMO-ESGO-ESTRO Consensus Conference on Endometrial Cancer: diagnosis, treatment and follow-up. Ann Oncol. 27(1), 16–41.
Gans SL, Pols MA, Stoker J, Boermeester MA, expert steering group (2015) Guideline for the diagnostic pathway in patients with acute abdominal pain, Dig Surg. 32(1), 23-31.
Paolantonio P, Rengo M, Ferrari R, Laghi A (2016) Multidetector CT in emergency radiology: acute and generalized non-traumatic abdominal pain, Br J Radiol. 89(1061), 20150859.
Karia M, Seager M, Rafique A, Sheth H (2017) The Diagnostic Utility and Clinical Impact of After-Hours CT Scans of the Abdomen and Pelvis Investigating Abdominal Pain, ScientificWorldJournal. 2017, 4028352.
Haldorsen I.S, Salvesen H.B (2016) What Is the Best Preoperative Imaging for Endometrial Cancer?, Current oncology reports. 18(4), 25-25.
Lin M.Y, Dobrotwir A, McNally O, Abu-Rustum NR, Narayan K (2018) Role of imaging in the routine management of endometrial cancer, Int J Gynaecol Obstet. 143 Suppl 2(Suppl 2), 109–117.
Trojano G, Olivieri C, Tinelli R, Damiani GR, Pellegrino A, Cicinelli E (2019) Conservative treatment in early stage endometrial cancer: a review. Acta Biomed. 90(4), 405-410.
Gressel G.M., Parkash V., L. Pal (2015) Management options and fertility-preserving therapy for premenopausal endometrial hyperplasia and early-stage endometrial cancer, Int J Gynaecol Obstet. 131(3), 234-9.
Rizzo S, et al (2018) Endometrial cancer: an overview of novelties in treatment and related imaging keypoints for local staging, Cancer Imaging. 18(1), 45.
Matteson, K.A., K. Robison, V.L. Jacoby (2018) Opportunities for Early Detection of Endometrial Cancer in Women with Postmenopausal Bleeding, JAMA Intern Med. 178(9), 1222-1223.
Banzhaf W, Nordin P, Keller RE, Francone FD (1998) Genetic programming: an introduction: on the automatic evolution of computer programs and its applications, Morgan Kaufmann Publishers Inc. 470.
Orlenko A, et al (2018) Considerations for automated machine learning in clinical metabolic profiling: Altered homocysteine plasma concentration associated with metformin exposure, Pacific Symposium on Biocomputing. 23, 460-471.
Ken Chang, et al (2019) Automatic assessment of glioma burden: a deep learning algorithm for fully automated volumetric and bidimensional measurement, Neuro-Oncology. 21(11), 1412–1422.
Cignini P, et al (2017) Preoperative work-up for definition of lymph node risk involvement in early stage endometrial cancer: 5-year follow-up, Updates Surg. 69(1), 75-82.
Ortoft G, et al (2013) Preoperative staging of endometrial cancer using TVS, MRI, and hysteroscopy, Acta Obstet Gynecol Scand. 92(5), 536-45.
Funding
This work is supported by the National Key Research and Development Program of China (No. 2018AAA0102100), the Scientific and Technological Innovation Leading Plan of High-tech Industry of Hunan Province (2020GK2021), the National Natural Science Foundation of China (No. 61902434) and the 111 project under grant (No. B18059).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no potential conflicts of interest.
Ethical approval
The study was approved by the Institutional Review Boards from respective institutions.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Li, D., Hu, R., Li, H. et al. Performance of automatic machine learning versus radiologists in the evaluation of endometrium on computed tomography. Abdom Radiol 46, 5316–5324 (2021). https://doi.org/10.1007/s00261-021-03210-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00261-021-03210-9