Blog
About

18
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Rayyan—a web and mobile app for systematic reviews

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Synthesis of multiple randomized controlled trials (RCTs) in a systematic review can summarize the effects of individual outcomes and provide numerical answers about the effectiveness of interventions. Filtering of searches is time consuming, and no single method fulfills the principal requirements of speed with accuracy. Automation of systematic reviews is driven by a necessity to expedite the availability of current best evidence for policy and clinical decision-making.

          We developed Rayyan ( http://rayyan.qcri.org), a free web and mobile app, that helps expedite the initial screening of abstracts and titles using a process of semi-automation while incorporating a high level of usability. For the beta testing phase, we used two published Cochrane reviews in which included studies had been selected manually. Their searches, with 1030 records and 273 records, were uploaded to Rayyan. Different features of Rayyan were tested using these two reviews. We also conducted a survey of Rayyan’s users and collected feedback through a built-in feature.

          Results

          Pilot testing of Rayyan focused on usability, accuracy against manual methods, and the added value of the prediction feature. The “taster” review (273 records) allowed a quick overview of Rayyan for early comments on usability. The second review (1030 records) required several iterations to identify the previously identified 11 trials. The “suggestions” and “hints,” based on the “prediction model,” appeared as testing progressed beyond five included studies. Post rollout user experiences and a reflexive response by the developers enabled real-time modifications and improvements. The survey respondents reported 40% average time savings when using Rayyan compared to others tools, with 34% of the respondents reporting more than 50% time savings. In addition, around 75% of the respondents mentioned that screening and labeling studies as well as collaborating on reviews to be the two most important features of Rayyan.

          As of November 2016, Rayyan users exceed 2000 from over 60 countries conducting hundreds of reviews totaling more than 1.6M citations. Feedback from users, obtained mostly through the app web site and a recent survey, has highlighted the ease in exploration of searches, the time saved, and simplicity in sharing and comparing include-exclude decisions. The strongest features of the app, identified and reported in user feedback, were its ability to help in screening and collaboration as well as the time savings it affords to users.

          Conclusions

          Rayyan is responsive and intuitive in use with significant potential to lighten the load of reviewers.

          Related collections

          Most cited references 8

          • Record: found
          • Abstract: not found
          • Article: not found

          Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.

            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Using text mining for study identification in systematic reviews: a systematic review of current approaches

            Background The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying relevant studies in an unbiased way for inclusion in systematic reviews both complex and time consuming. Text mining has been offered as a potential solution: through automating some of the screening process, reviewer time can be saved. The evidence base around the use of text mining for screening has not yet been pulled together systematically; this systematic review fills that research gap. Focusing mainly on non-technical issues, the review aims to increase awareness of the potential of these technologies and promote further collaborative research between the computer science and systematic review communities. Methods Five research questions led our review: what is the state of the evidence base; how has workload reduction been evaluated; what are the purposes of semi-automation and how effective are they; how have key contextual problems of applying text mining to the systematic review field been addressed; and what challenges to implementation have emerged? We answered these questions using standard systematic review methods: systematic and exhaustive searching, quality-assured data extraction and a narrative synthesis to synthesise findings. Results The evidence base is active and diverse; there is almost no replication between studies or collaboration between research teams and, whilst it is difficult to establish any overall conclusions about best approaches, it is clear that efficiencies and reductions in workload are potentially achievable. On the whole, most suggested that a saving in workload of between 30% and 70% might be possible, though sometimes the saving in workload is accompanied by the loss of 5% of relevant studies (i.e. a 95% recall). Conclusions Using text mining to prioritise the order in which items are screened should be considered safe and ready for use in ‘live’ reviews. The use of text mining as a ‘second screener’ may also be used cautiously. The use of text mining to eliminate studies automatically should be considered promising, but not yet fully proven. In highly technical/clinical areas, it may be used with a high degree of confidence; but more developmental and evaluative work is needed in other disciplines. Electronic supplementary material The online version of this article (doi:10.1186/2046-4053-4-5) contains supplementary material, which is available to authorized users.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              A method for assessing the quality of a randomized control trial.

              A system has been constructed to evaluate the design, implementation, and analysis of randomized control trials (RCT). The degree of quadruple blinding (the randomization process, the physicians and patients as to therapy, and the physicians as to ongoing results) is considered to be the most important aspect of any trial. The analytic techniques are scored with the same emphasis as is placed on the control of bias in the planning and implementation of the studies. Description of the patient and treatment materials and the measurement of various controls of quality have less weight. An index of quality of a RCT is proposed with its pros and cons. If published papers were to approximate these principles, there would be a marked improvement in the quality of randomized control trials. Finally, a reasonable standard design and conduct of trials will facilitate the interpretation of those with conflicting results and help in making valid combinations of undersized trials.
                Bookmark

                Author and article information

                Contributors
                mouzzani@qf.org.qa
                hhammady@qf.org.qa
                zbysfedorowicz@gmail.com
                aelmagarmid@qf.org.qa
                Journal
                Syst Rev
                Syst Rev
                Systematic Reviews
                BioMed Central (London )
                2046-4053
                5 December 2016
                5 December 2016
                2016
                : 5
                Affiliations
                [1 ]Qatar Computing Research Institute, HBKU, Doha, Qatar
                [2 ]Cochrane Bahrain, Awali, Bahrain
                Article
                384
                10.1186/s13643-016-0384-4
                5139140
                27919275
                © The Author(s) 2016

                Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

                Categories
                Methodology
                Custom metadata
                © The Author(s) 2016

                Public health

                automation, systematic reviews, evidence-based medicine

                Comments

                Comment on this article