Lo, Pechin25; Ginneken, Bram van3; Reinhardt, Joseph M.26; Yavarna, Tarunashree26; Jong, Pim A. de3; Irving, Benjamin27; Fetita, Catalin6; Ortner, Margarete6; Pinho, Rômulo7; Sijbers, Jan28; Feuerstein, Marco9; Fabijanska, Anna29; Bauer, Christian11; Beichel, Reinhard26; Mendoza, Carlos S.30; Wiemker, Rafael13; Lee, Jaesung14; Reeves, Anthony P.14; Born, Silvia15; Weinheimer, Oliver16; Rikxoort, Eva M. van17; Tschirren, Juerg18; Mori, Ken31; Odry, Benjamin20; Naidich, David P.21; Hartmann, Ieneke22; Hoffman, Eric A.26; Prokop, Mathias23; Pedersen, Jesper H.24; de Bruijne, Marleen25
1 Department of Computer Science, Faculty of Science, Københavns Universitet2 The Image Section, Department of Computer Science, Faculty of Science, Københavns Universitet3 University Medical Center Utrecht4 University of Iowa5 University College London6 Institut TELECOM/Telecom SudParis7 University of Lyon8 University of Antwerp9 Technische Universität München10 Lodz University of Technology11 Graz University of Technology12 Universidad de Sevilla13 Philips Research Laboratories Hamburg14 Cornell University15 Universität Leipzig16 Johannes-Gutenberg-University17 Radboud University Nijmegen Medical Centre18 VIDA Diagnostics19 Nagoya University20 Siemens Corporation21 New York University Medical Center22 Erasmus MC University Medical Center23 Radboud University NijmegenMedical Centre24 Copenhagen University Hospital, Rigshospitalet25 Department of Computer Science, Faculty of Science, Københavns Universitet26 University of Iowa27 University College London28 University of Antwerp29 Lodz University of Technology30 Universidad de Sevilla31 Nagoya University
This paper describes a framework for establishing a reference airway tree segmentation, which was used to quantitatively evaluate 15 different airway tree extraction algorithms in a standardized manner. Because of the sheer difficulty involved in manually constructing a complete reference standard from scratch, we propose to construct the reference using results from all algorithms that are to be evaluated. We start by subdividing each segmented airway tree into its individual branch segments. Each branch segment is then visually scored by trained observers to determine whether or not it is a correctly segmented part of the airway tree. Finally, the reference airway trees are constructed by taking the union of all correctly extracted branch segments. Fifteen airway tree extraction algorithms from different research groups are evaluated on a diverse set of 20 chest computed tomography (CT) scans of subjects ranging from healthy volunteers to patients with severe pathologies, scanned at different sites, with different CT scanner brands, models, and scanning protocols. Three performance measures covering different aspects of segmentation quality were computed for all participating algorithms. Results from the evaluation showed that no single algorithm could extract more than an average of 74% of the total length of all branches in the reference standard, indicating substantial differences between the algorithms. A fusion scheme that obtained superior results is presented, demonstrating that there is complementary information provided by the different algorithms and there is still room for further improvements in airway segmentation algorithms.
I E E E Transactions on Medical Imaging, 2012, Vol 31, Issue 11