9
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Repeatability and Reproducibility of Radiomic Features: A Systematic Review

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Purpose:

          An ever-growing number of predictive models used to inform clinical decision making have included quantitative, computer-extracted imaging biomarkers, or “radiomic features.” Broadly generalizable validity of radiomics-assisted models may be impeded by concerns about reproducibility. We offer a qualitative synthesis of 41 studies that specifically investigated the repeatability and reproducibility of radiomic features, derived from a systematic review of published peer-reviewed literature.

          Methods and Materials:

          The PubMed electronic database was searched using combinations of the broad Haynes and Ingui filters along with a set of text words specific to cancer, radiomics (including texture analyses), reproducibility, and repeatability. This review has been reported in compliance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. From each full-text article, information was extracted regarding cancer type, class of radiomic feature examined, reporting quality of key processing steps, and statistical metric used to segregate stable features.

          Results:

          Among 624 unique records, 41 full-text articles were subjected to review. The studies primarily addressed non-small cell lung cancer and oropharyngeal cancer. Only 7 studies addressed in detail every methodologic aspect related to image acquisition, preprocessing, and feature extraction. The repeatability and reproducibility of radiomic features are sensitive at various degrees to processing details such as image acquisition settings, image reconstruction algorithm, digital image preprocessing, and software used to extract radiomic features. First-order features were overall more reproducible than shape metrics and textural features. Entropy was consistently reported as one of the most stable first-order features. There was no emergent consensus regarding either shape metrics or textural features; however, coarseness and contrast appeared among the least reproducible.

          Conclusions:

          Investigations of feature repeatability and reproducibility are currently limited to a small number of cancer types. Reporting quality could be improved regarding details of feature extraction software, digital image manipulation (preprocessing), and the cutoff value used to distinguish stable features.

          Summary

          We offer a qualitative synthesis of 41 studies that specifically investigated the repeatability and reproducibility of radiomic features. The repeatability and reproducibility of radiomic features are sensitive at various degrees to image quality and to software used to extract radiomic features. Investigations of feature repeatability and reproducibility are currently limited to a small number of cancer types. No consensus was found regarding the most repeatable and reproducible features with respect to different settings.

          Related collections

          Most cited references 50

          • Record: found
          • Abstract: found
          • Article: not found

          A concordance correlation coefficient to evaluate reproducibility.

           Aigu L. Lin (1989)
          A new reproducibility index is developed and studied. This index is the correlation between the two readings that fall on the 45 degree line through the origin. It is simple to use and possesses desirable properties. The statistical properties of this estimate can be satisfactorily evaluated using an inverse hyperbolic tangent transformation. A Monte Carlo experiment with 5,000 runs was performed to confirm the estimate's validity. An application using actual data is given.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM.

            Reliability, the consistency of a test or measurement, is frequently quantified in the movement sciences literature. A common metric is the intraclass correlation coefficient (ICC). In addition, the SEM, which can be calculated from the ICC, is also frequently reported in reliability studies. However, there are several versions of the ICC, and confusion exists in the movement sciences regarding which ICC to use. Further, the utility of the SEM is not fully appreciated. In this review, the basics of classic reliability theory are addressed in the context of choosing and interpreting an ICC. The primary distinction between ICC equations is argued to be one concerning the inclusion (equations 2,1 and 2,k) or exclusion (equations 3,1 and 3,k) of systematic error in the denominator of the ICC equation. Inferential tests of mean differences, which are performed in the process of deriving the necessary variance components for the calculation of ICC values, are useful to determine if systematic error is present. If so, the measurement schedule should be modified (removing trials where learning and/or fatigue effects are present) to remove systematic error, and ICC equations that only consider random error may be safely used. The use of ICC values is discussed in the context of estimating the effects of measurement error on sample size, statistical power, and correlation attenuation. Finally, calculation and application of the SEM are discussed. It is shown how the SEM and its variants can be used to construct confidence intervals for individual scores and to determine the minimal difference needed to be exhibited for one to be confident that a true change in performance of an individual has occurred.
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Dangers of using "optimal" cutpoints in the evaluation of prognostic factors.

                Bookmark

                Author and article information

                Journal
                7603616
                4036
                Int J Radiat Oncol Biol Phys
                Int. J. Radiat. Oncol. Biol. Phys.
                International journal of radiation oncology, biology, physics
                0360-3016
                1879-355X
                23 July 2019
                05 June 2018
                15 November 2018
                12 August 2019
                : 102
                : 4
                : 1143-1158
                Affiliations
                [* ] Department of Radiation Oncology, MAASTRO Clinic, Maastricht, The Netherlands
                [] School for Oncology and Developmental Biology (GROW), Maastricht University, Maastricht, The Netherlands
                [] Moffitt Cancer Center, Tampa, Florida
                Author notes
                Reprint requests to: Alberto Traverso, PhD, Department of Radiation, Oncology, MAASTRO Clinic, Dr Tanslaan 12, 6229ET, Maastricht, The Netherlands. Tel: +31615376849; alberto.traverso@ 123456maastro.nl
                Article
                NIHMS1032249
                10.1016/j.ijrobp.2018.05.053
                6690209
                30170872

                This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/).

                Categories
                Article

                Oncology & Radiotherapy

                Comments

                Comment on this article