1. The complete title of one (or more) paper(s) published in the open literature describing the work that the author claims describes a human-competitive result; Our entry consists of 2 journal articles published in high-quality international peer-reviewed journals on medical imaging and on physics in medicine. Article A) "Machine learning for the prediction of pseudorealistic pediatric abdominal phantoms for radiation dose reconstruction" Article B) "Surrogate-free machine learning-based organ dose reconstruction for pediatric abdominal radiotherapy" ----------------------------------------------------------------------------------------------------------------------- 2. The name, complete physical mailing address, e-mail address, and phone number of EACH author of EACH paper(s); We report the authors for the articles A and B in the same order as they appear on the papers. Article A) Marco Virgolin ― mail: Centrum Wiskunde & Informatica (CWI), P.O. Box 94079, 1090 GB Amsterdam, the Netherlands** ― email: marco.virgolin@cwi.nl, ― phone: +31 633 852 233 Ziyuan Wang ― mail: Department of Radiation Oncology, Amsterdam UMC locatie AMC, Meibergdreef 9 1105 AZ Amsterdam, NL ― email: z.wang@amsterdamumc.nl ― phone: +31 644 861 780 Tanja Alderliesten ― mail: Department of Radiation Oncology, Leids Universitair Medisch Centrum (LUMC), P.O. Box 9600, 2300 RC Leiden, the Netherlands ― email: t.alderliesten@lumc.nl ― phone: +31 71 526 5539 Peter A.N. Bosman ― mail: Centrum Wiskunde & Informatica (CWI), P.O. Box 94079, 1090 GB Amsterdam, the Netherlands ― email: peter.bosman@cwi.nl ― phone: +31(0)20 592 4265 Article B) Marco Virgolin ― mail: Centrum Wiskunde & Informatica (CWI), P.O. Box 94079, 1090 GB Amsterdam, the Netherlands** ― email: marco.virgolin@cwi.nl, ― phone: +31 633 852 233 Ziyuan Wang ― mail: Department of Radiation Oncology, Amsterdam UMC locatie AMC, Meibergdreef 9 1105 AZ Amsterdam, NL ― email: z.wang@amsterdamumc.nl ― phone: +31 644 861 780 Brian V. Balgobind ― mail: Department of Radiation Oncology, Amsterdam UMC locatie AMC, Meibergdreef 9 1105 AZ Amsterdam, the Netherlands ― email: b.v.balgobind@amsterdamumc.nl ― phone: +31 20 566 9111 Irma W.E.M. van Dijk ― mail: Department of Radiation Oncology, Amsterdam UMC locatie AMC, Meibergdreef 9 1105 AZ Amsterdam, the Netherlands ― email: i.w.vandijk@amsterdamumc.nl ― phone: +31 20 566 9111 Jan Wiersma ― mail: Department of Radiation Oncology, Amsterdam UMC locatie AMC, Meibergdreef 9 1105 AZ Amsterdam, the Netherlands ― email: j.wiersma@amsterdamumc.nl ― phone: +31 20 566 9111 Petra S. Kroon ― mail: Department of Radiotherapy, University Medical Center Utrecht (UMCU), P.O. Box 85500, 3508 GA Utrecht, the Netherlands ― email: p.s.kroon-3@umcutrecht.nl ― phone: +31 88 755 5555 Geert O. Janssens ― mail: Department of Radiotherapy, University Medical Center Utrecht (UMCU), P.O. Box 85500, 3508 GA Utrecht, the Netherlands ― email: g.o.r.janssens@umcutrecht.nl ― phone: +31 88 755 5555 Marcel van Herk ― mail: Manchester Cancer Research Centre, the University of Manchester, 555 Wilmslow Road, M20 4GJ Manchester, United Kingdom ― email: marcel.vanherk@manchester.ac.uk ― phone: +44 161 306 0800 David C. Hodgson ― mail: Princess Margaret Cancer Center, 610 University Avenue Toronto, ON M5G 2M9, Canada ― email: david.hodgson@rmp.uhn.ca ― phone: +1 416-946-2000 Lorna Zadravec Zaletel ― mail: Division of Radiotherapy, Institute of Oncology, Zaloška cesta 2, 1000 Ljubljana, Slovenia ― email: lzaletel@onko-i.si ― phone: +386 1 5879 110 Coen R.N. Rasch ― mail: Department of Radiation Oncology, Leids Universitair Medisch Centrum (LUMC), P.O. Box 9600, 2300 RC Leiden, the Netherlands ― email: c.r.n.rasch@lumc.nl ― phone: +31 71 526 9111 Arjan Bel ― mail: Department of Radiation Oncology, Amsterdam UMC locatie AMC, Meibergdreef 9 1105 AZ Amsterdam, NL ― email: a.bel@amsterdamumc.nl ― phone: +31 20 566 9111 Peter A.N. Bosman ― mail: Centrum Wiskunde & Informatica (CWI), P.O. Box 94079, 1090 GB Amsterdam, the Netherlands ― email: peter.bosman@cwi.nl ― phone: +31(0)20 592 4265 Tanja Alderliesten ― mail: Department of Radiation Oncology, Leids Universitair Medisch Centrum (LUMC), P.O. Box 9600, 2300 RC Leiden, the Netherlands ― email: t.alderliesten@lumc.nl ― phone: +31 71 526 5539 **Marco is currently with Chalmers University of Technology, Gothenburg, Sweden; However, the research work was conducted at CWI, and he will start a new position at CWI again, later this year. ----------------------------------------------------------------------------------------------------------------------- 3. The name of the corresponding author (i.e., the author to whom notices will be sent concerning the competition) Marco Virgolin ----------------------------------------------------------------------------------------------------------------------- 4. The abstract of the paper(s); Article A) Purpose: Current phantoms used for the dose reconstruction of long-term childhood cancer survivors lack individualization. We design a method to predict highly individualized abdominal three-dimensional (3-D) phantoms automatically. Approach: We train machine learning (ML) models to map (2-D) patient features to 3-D organ-at-risk (OAR) metrics upon a database of 60 pediatric abdominal computed tomographies with liver and spleen segmentations. Next, we use the models in an automatic pipeline that outputs a personalized phantom given the patient’s features, by assembling 3-D imaging from the database. A step to improve phantom realism (i.e., avoid OAR overlap) is included. We compare five ML algorithms, in terms of predicting OAR left–right (LR), anterior–posterior (AP), inferior–superior (IS) positions, and surface Dice–Sørensen coefficient (sDSC). Furthermore, two existing human-designed phantom construction criteria and two additional control methods are investigated for comparison. Results: Different ML algorithms result in similar test mean absolute errors: ∼8mm for liver LR, IS, and spleen AP, IS; ∼5mm for liver AP and spleen LR; ∼80% for abdomen sDSC; and ∼60% to 65% for liver and spleen sDSC. One ML algorithm (GP-GOMEA) significantly performs the best for 6/9 metrics. The control methods and the human-designed criteria in particular perform generally worse, sometimes substantially (+5-mm error for spleen IS, −10% sDSC for liver). The automatic step to improve realism generally results in limited metric accuracy loss, but fails in one case (out of 60). Conclusion: Our ML-based pipeline leads to phantoms that are significantly and substantially more individualized than currently used human-designed criteria. Article B) To study radiotherapy-related adverse effects, detailed dose information (3D distribution) is needed for accurate dose-effect modeling. For childhood cancer survivors who underwent radiotherapy in the pre-CT era, only 2D radiographs were acquired, thus 3D dose distributions must be reconstructed from limited information. State-of-the-art methods achieve this by using 3D surrogate anatomies. These can however lack personalization and lead to coarse reconstructions. We present and validate a surrogate-free dose reconstruction method based on Machine Learning (ML). Abdominal planning CTs (n=142) of recently-treated childhood cancer patients were gathered, their organs at risk were segmented, and 300 artificial Wilms' tumor plans were sampled automatically. Each artificial plan was automatically emulated on the 142 CTs, resulting in 42,600 3D dose distributions from which dose-volume metrics were derived. Anatomical features were extracted from digitally reconstructed radiographs simulated from the CTs to resemble historical radiographs. Further, patient and radiotherapy plan features typically available from historical treatment records were collected. An evolutionary ML algorithm was then used to link features to dose-volume metrics. Besides 5-fold cross validation, a further evaluation was done on an independent dataset of five CTs each associated with two clinical plans. Cross-validation resulted in mean absolute errors ≤ 0.6 Gy for organs completely inside or outside the field. For organs positioned at the edge of the field, mean absolute errors ≤ 1.7 Gy for Dmean, ≤ 2.9 Gy for D2cc, and ≤ 13% for V5Gy and V10Gy, were obtained, without systematic bias. Similar results were found for the independent dataset. To conclude, we proposed a novel organ dose reconstruction method that uses ML models to predict dose-volume metric values given patient and plan features. Our approach is not only accurate, but also efficient, as the setup of a surrogate is no longer needed. ----------------------------------------------------------------------------------------------------------------------- 5. A list containing one or more of the eight letters (A, B, C, D, E, F, G, or H) that correspond to the criteria (see above) that the author claims that the work satisfies (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. (D) The result is publishable in its own right as a new scientific result independent of the fact that the result was mechanically created. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. ----------------------------------------------------------------------------------------------------------------------- 6. A statement stating why the result satisfies the criteria that the contestant claims (see examples of statements of human-competitiveness as a guide to aid in constructing this part of the submission) To understand part of the terminology used here, we recommend reading our answer to point 9 first, where we provide a description of the application. (B) Dose reconstruction methods, which involve (i) surrogate anatomies being generated (in part by hand, in part automatically) and (ii) models of anatomy similarity between patient features and available phantoms, have been published since early 2000, up to now. These methods, in particular the two methods we compare with, are very much used in practice by epidemiology researchers, to model how oncology treatment, including radiation oncology, causes adverse effects. Unfortunately, different types of dose reconstruction methods are rarely, if ever, compared on the same footing. This has various reasons, most notably the necessity of full cooperation of the inventing institutes, in particular for various manual steps that are required. For such reasons, we also cannot submit a head-to-head comparison at this time. However, we do submit a result that is based on the next best thing: a comparison to a simulation of the dose reconstruction process that includes models that closely replicate the manual steps as typically carried out by an institute. In article A, we show that GP-found models outperform two famous existing approaches that are currently used to perform state-of-the-art dose reconstructions. The gains are, at times, really substantial. In article B, we show that a new type of (GP) models can be created to skip the use of surrogate anatomies (i.e., the phantoms) altogether, by directly linking features of the patient and of the treatment plan to dosimetric outcomes of interest. This greatly simplifies the dose reconstruction process, since one no longer needs to set-up a simulation environment where the phantom is irradiated, to collect dose estimations. Moreover, again, the models found by GP perform better than a (simulated) famous partially-manual phantom-construction approach we compared with. (D) The approach itself, including the results that show it can outperform simulations of existing, partially manual approaches, is already published (see 7). Moreover, it was published in a non-EA journal, but rather a medical physics journal, underlining the fact that the results in the application field that we have obtained are already publishable in their own right. On top of this, the authors of the two alternative approaches have agreed that our result is worth publishing in its own right in a head-to-head validation study to be submitted to a medical journal and have agreed to spending the required effort for this. (E) (Phantom-based) dose reconstruction methods have been published over many years, claiming progressive improvement. Our result is compared to two of these. The first one is from early 2000. This is the relatively simpler one, but it is also extensively adopted: an update paper was published by the group that coined the method in 2019, where over 120 radiation epidemiology studies that have relied on the method are cited. The second one is from 2014 and uses more realistic phantoms and a more involved (yet, still relatively simple) model for surrogate selection, and can be considered to be a contender of the first mentioned approach. The models we find with GP allow us to obtain (at times, far) better matching phantoms than the existing methods (article A), or even dosimetric estimations directly without needing phantoms (article B), compared to (our in-house simulated versions of) the dose reconstructions performed by the existing methods. ----------------------------------------------------------------------------------------------------------------------- 7. A full citation of the paper (that is, author names; title, publication date; name of journal, conference, or book in which article appeared; name of editors, if applicable, of the journal or edited book; publisher name; publisher city; page numbers, if applicable); 7.A M Virgolin and Z Wang, T Alderliesten, PAN Bosman; "Machine learning for the prediction of pseudorealistic pediatric abdominal phantoms for radiation dose reconstruction", July 30 2020; Journal of Medical Imaging; SPIE; 7(4), p.046501. (DOI: https://doi.org/10.1117/1.JMI.7.4.046501) 7.B M Virgolin, Z Wang, B V Balgobind, I W E M van Dijk, J Wiersma, P S Kroon, G O Janssens, M van Herk, D C Hodgson, L Zadravec Zaletel, C R N Rasch, A Bel, P A N Bosman, T Alderliesten; "Surrogate-free machine learning-based organ dose reconstruction for pediatric abdominal radiotherapy", December 8 2020; Physics in Medicine & Biology; IOP Publishing; Bristol; 65(24), p.245021. (DOI: https://doi.org/10.1088/1361-6560/ab9fcc) ----------------------------------------------------------------------------------------------------------------------- 8. A statement either that "any prize money, if any, is to be divided equally among the co-authors" OR a specific percentage breakdown as to how the prize money, if any, is to be divided among the co-authors; Prize money, if any, is to be equally divided over the two main collaborating research groups at CWI and Amsterdam UMC, represented by Tanja Alderliesten (Amsterdam UMC (at the time of submission)) and Peter Bosman (CWI). ----------------------------------------------------------------------------------------------------------------------- 9. A statement stating why the authors expect that their entry would be the "best" Our entry combines cutting-edge algorithmic research in Genetic Programming with a crucial societal application, i.e., childhood cancer radiation treatment. We believe that our entry can be the best one for the following reasons: - The articles constituting our entry have been published in prestigious, peer-reviewed international journals of medical imaging (article A, Journal of Medical Imaging) and medical physics (article B, Physics in Medicine and Biology). - The GP algorithm that our entry relies upon (GP-GOMEA, a model-based GP algorithm that makes use of information theory to build a model of potential building blocks in the population), was firstly presented with a full-paper at GECCO 2017, and subsequently refined in a journal article published in Evolutionary Computation in 2020; both are recognized as top venues. - This application has direct societal relevance. A description follows. Radiation treatment is a double-edged sword: doctors need to irradiate the tumor as much as possible, without causing excessive damage to the surrounding organs. For childhood cancer survivors who underwent radiation treatment before the '90s, no 3D imaging was acquired at the time (but only 2D radiographs), hence we lack information on the 3D radiation dose distribution that they were subjected to. This information is crucial, however, to understand the late adverse effects of radiation treatment, and to allow us to improve today's childhood cancer treatment. To tackle this problem, researchers have been developing so-called "phantoms", i.e., 3D (typically virtual) surrogate anatomies that are used in place of the (missing) 3D imaging of the patient; the treatment is simulated on the phantom, and the 3D dose distribution is estimated. This process is called "dose reconstruction". According to the literature, the accuracy of dose reconstruction *mostly* depends on how well the phantom matches the body of the patient (i.e., errors for the other steps of the dose reconstruction process are smaller). Matching is based on the few features available for the patient, using models hand-crafted by experts. - Mathematical or algorithmic models used in current dose reconstruction approaches, upon the basis of which phantoms are created and selected to be surrogates of a patient, have always been hand-crafted. To the best of our knowledge, we are the first to propose models found by an algorithm. In our work, we show that GP (GP-GOMEA in particular) finds models that not only outperform human-designed ones (and at times substantially so), they also often outperform those found by other machine learning algorithms, such as Elastic Net and Random Forest. In summary, our entry entails that, with GP, we can outperform hand-crafted methods for radiation dose reconstruction that are adopted in hundreds of epidemiology studies. Not only that, GP models are suitable for interpretation (we provide examples of this in paper A) which can give medical doctors a possibility to inspect the models for sensibleness, and gain trust in their use. ----------------------------------------------------------------------------------------------------------------------- 10. An indication of the general type of genetic or evolutionary computation used, such as GA (genetic algorithms), GP (genetic programming), ES (evolution strategies), EP (evolutionary programming), LCS (learning classifier systems), GI (genetic improvement), GE (grammatical evolution), GEP (gene expression programming), DE (differential evolution). GP (genetic programming) ----------------------------------------------------------------------------------------------------------------------- 11. The date of publication of each paper. If the date of publication is not on or before the deadline for submission, but instead, the paper has been unconditionally accepted for publication and is “in press” by the deadline for this competition, the entry must include a copy of the documentation establishing that the paper meets the "in press" requirement. Article A) Published on July 30 2020 in the Journal of Medical Imaging, DOI: https://doi.org/10.1117/1.JMI.7.4.046501 Article B) Published on December 8 2020 in Physics in Medicine & Biology, DOI: https://doi.org/10.1088/1361-6560/ab9fcc