8
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Clinical brain PET research must embrace multi-centre collaboration and data sharing or risk its demise

      letter

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Dear Editor, Genetics in the early 2000s consisted primarily of studies in small samples from individual research centres. Following the successful initial identification of very rare genetic variants which cause large effects, the search continued for individual genes which might explain a substantial proportion of the phenotypic variance in the wider population. However, it soon became clear that such genes simply do not exist, and that nearly all conclusions of the latter studies were incorrect [1]. To rise to the challenge, the field collectively moved towards collaborative research, yielding multi-centre sample sizes of up to tens of thousands, and genetics is now widely considered to produce robust scientific results [2]. During recent years, researchers within several fields of neuroimaging research, particularly MRI and fMRI, have begun to make this transition to data sharing and collaborative research, facilitated by technical developments in data handling and analysis [3–5]. Just like genetics, clinical positron emission tomography (PET) research has provided answers to numerous research questions across several domains. For example, the dopamine transporter shows clear decreases in Parkinson’s disease, dopamine synthesis capacity is increased in schizophrenia, and mony brain neurotransmission proteins show distinct decreases across the lifespan. However, for the well-established tracers and targets, it can be argued that we have already picked most of the low-hanging fruit. In the continued quest to break new ground, it is likely that most of the studied effects will be small, which means that large sample sizes will be needed to reach the threshold of statistical significance [6]. If we instead continue to use small sample sizes to search for subtle true effects, we run the risk of fooling ourselves into seeing “patterns” in what is really noise, leading to reporting of spurious effects. Further, with small samples, even when we correctly identify true effects as significant, our effect size estimates will be biased upward [6, 7]. We acknowledge that there will always be an important role for small exploratory studies for generating new hypotheses, but the subsequent confirmation and quantitative description of these hypotheses simply requires higher standards of evidence to move the field forward. Importantly, the problem of unreliable findings is by no means restricted to PET research, as has recently been evidenced by “replication crises” in other fields such as psychology, economics, and preclinical drug discovery research [8–12]. Unfortunately, large samples in the field of PET are unattainable for many individual research centres, owing to the high cost and technical difficulty of the method [13]. Traditionally, the proposed remedy to the issue of small sample size studies has been to perform meta-analyses to gain an overall, field-wide estimate of the studied effect. However, traditional meta-analysis has its own set of limitations. If the individual studies entered into a meta-analysis consist of biased effect size estimates, then the overall effect size will also be misleading [14–16]. It is also not possible to control for confounders, or to account for differences in outcome measures between studies [17]. One solution is to instead make use of the original data points collected by individual research centres. In the previous issue of EJNMMI, we report the results of such a multi-centre collaboration, or “mega-analysis” (Tuisku et al. [18]). By synthesising translocator protein (TSPO)–binding data from three different centres, effects were shown for age, BMI, and sex on TSPO, some of which were not evident in previous studies using smaller samples. Apart from informing the design and interpretation of TSPO PET studies, the results may also open up new avenues of research into the biological role of TSPO. Multi-centre collaboration and data sharing entail certain considerations. With more researchers working on the same problem, the risk for differences in opinion regarding outcome measures, statistical analyses, and even the nature of the hypotheses increases [19]. We have found it useful to formally make these decisions in advance. A Memorandum of Understanding (MoU) may serve as an initial step, containing rules regarding data handling, the general outline of the analysis, as well as author number and order. This document can then be complemented by a specific pre-registration protocol for the analysis, detailing how data will be synthesised, which hypotheses will be tested, which statistical models will be used to make inference, etc. [19, 20]. When all authors have come to an agreement on the content, the protocol can be locked and uploaded to a date-stamped public repository. In the ensuing analysis, deviations from the pre-registration are still possible, provided that they are reported in addition to the original protocol. Importantly, sharing of individual participant outcome measures, such as binding values, is only the first step. By using data in as raw a form as possible, the data processing in the combined analysis can be made more homogeneous. This can be achieved by using either a centralised analysis, or by using reproducible, open-source tools for which all procedures are scripted and can be run in an identical fashion [21, 22]. Hence, in the case of PET studies, the sharing of time activity curves is better than sharing of binding outcomes, while raw image data is better still, allowing for homogeneous data analysis all the way from image processing to pharmacokinetic modelling [23]. An additional measure to minimise between-centre differences would then be to use harmonised protocols for data collection. The sharing of raw image data has historically been challenging, as storage and processing of files can differ between, or even within research groups. These complications are effectively resolved by the recently developed Brain Imaging Data Structure (BIDS) [3]. BIDS consists of a set of standards for storing brain imaging data, such that preprocessing and analysis can be performed in a standardised fashion, further simplified by BIDS Apps [4]. Further, the OpenNeuro repository allows for open sharing of neuroimaging data according to the BIDS standard, and is already in wide use by the MRI, EEG, and MEG research communities. Today, there are also a limited number of PET measurements available on this platform (e.g. https://openneuro.org/datasets/ds001421/versions/1.0.1). At the NeuroReceptor Mapping conference in London 2018, a proposal to set up an open PET data sharing archive using the BIDS standard received unanimous support. By sharing individual participant data, regulatory aspects regarding data integrity come into play. The principle for data sharing adopted by the EU commission is that of “as open as possible, as closed as necessary” [24]. Contrary to common belief, the General Data Protection Regulation (GDPR) is designed to facilitate sharing of research data and collaboration, provided that sufficient steps have been taken to perform de-identification. A full interpretation of the implications of this new legislation is currently underway for many research centres/countries, and at present local guidelines may differ. Either way, we encourage researchers to begin as early as possible to ask research participants for permission for open sharing of research data for ongoing and planned PET studies, in order to ensure that future legal obstacles can be minimised. Efforts are underway to assist researchers in this matter, by creating template forms for informed consent which comply with all regulatory statutes (https://open-brain-consent.readthedocs.io). Within the PET brain imaging field, we are now at a crossroads. Will we continue to work solely within individual research centres, using small samples to yield incomplete, or even misleading, results from confirmatory studies; or will we make the transition to multi-centre collaboration and data sharing as exemplified by the genetics community? We hope for the latter, sooner rather than later, in order to ensure the continued success of PET research in driving our understanding of the biochemical basis of brain function and dysfunction.

          Related collections

          Most cited references8

          • Record: found
          • Abstract: found
          • Article: found
          Is Open Access

          BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods

          The rate of progress in human neurosciences is limited by the inability to easily apply a wide range of analysis methods to the plethora of different datasets acquired in labs around the world. In this work, we introduce a framework for creating, testing, versioning and archiving portable applications for analyzing neuroimaging data organized and described in compliance with the Brain Imaging Data Structure (BIDS). The portability of these applications (BIDS Apps) is achieved by using container technologies that encapsulate all binary and other dependencies in one convenient package. BIDS Apps run on all three major operating systems with no need for complex setup and configuration and thanks to the comprehensiveness of the BIDS standard they require little manual user input. Previous containerized data processing solutions were limited to single user environments and not compatible with most multi-tenant High Performance Computing systems. BIDS Apps overcome this limitation by taking advantage of the Singularity container technology. As a proof of concept, this work is accompanied by 22 ready to use BIDS Apps, packaging a diverse set of commonly used neuroimaging algorithms.
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Reproducibility.

              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              On the reproducibility of meta-analyses: six practical recommendations

              Background Meta-analyses play an important role in cumulative science by combining information across multiple studies and attempting to provide effect size estimates corrected for publication bias. Research on the reproducibility of meta-analyses reveals that errors are common, and the percentage of effect size calculations that cannot be reproduced is much higher than is desirable. Furthermore, the flexibility in inclusion criteria when performing a meta-analysis, combined with the many conflicting conclusions drawn by meta-analyses of the same set of studies performed by different researchers, has led some people to doubt whether meta-analyses can provide objective conclusions. Discussion The present article highlights the need to improve the reproducibility of meta-analyses to facilitate the identification of errors, allow researchers to examine the impact of subjective choices such as inclusion criteria, and update the meta-analysis after several years. Reproducibility can be improved by applying standardized reporting guidelines and sharing all meta-analytic data underlying the meta-analysis, including quotes from articles to specify how effect sizes were calculated. Pre-registration of the research protocol (which can be peer-reviewed using novel ‘registered report’ formats) can be used to distinguish a-priori analysis plans from data-driven choices, and reduce the amount of criticism after the results are known. Summary The recommendations put forward in this article aim to improve the reproducibility of meta-analyses. In addition, they have the benefit of “future-proofing” meta-analyses by allowing the shared data to be re-analyzed as new theoretical viewpoints emerge or as novel statistical techniques are developed. Adoption of these practices will lead to increased credibility of meta-analytic conclusions, and facilitate cumulative scientific knowledge.
                Bookmark

                Author and article information

                Contributors
                simon.cervenka@ki.se
                Journal
                Eur J Nucl Med Mol Imaging
                Eur. J. Nucl. Med. Mol. Imaging
                European Journal of Nuclear Medicine and Molecular Imaging
                Springer Berlin Heidelberg (Berlin/Heidelberg )
                1619-7070
                1619-7089
                12 November 2019
                12 November 2019
                2020
                : 47
                : 2
                : 502-504
                Affiliations
                [1 ]GRID grid.4714.6, ISNI 0000 0004 1937 0626, Centre for Psychiatry Research, Department of Clinical Neuroscience, , Karolinska Institutet and Stockholm County, ; Stockholm, Sweden
                [2 ]GRID grid.1374.1, ISNI 0000 0001 2097 1371, Turku PET Centre, , University of Turku, ; Turku, Finland
                [3 ]GRID grid.410552.7, ISNI 0000 0004 0628 215X, Turku PET Centre, , Turku University Hospital, ; Turku, Finland
                [4 ]GRID grid.47100.32, ISNI 0000000419368710, PET Center, Department of Radiology and Biomedical Imaging, , Yale University, ; New Haven, USA
                [5 ]GRID grid.47100.32, ISNI 0000000419368710, Department of Psychiatry, , Yale University, ; New Haven, USA
                Author information
                http://orcid.org/0000-0001-8103-6977
                Article
                4541
                10.1007/s00259-019-04541-y
                6974527
                31713656
                3c751d83-3e53-4909-879e-3fc27cb8dcd2
                © The Author(s) 2019

                Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

                History
                : 10 September 2019
                : 16 September 2019
                Categories
                Letter to the Editor
                Custom metadata
                © Springer-Verlag GmbH Germany, part of Springer Nature 2020

                Radiology & Imaging
                Radiology & Imaging

                Comments

                Comment on this article