174
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Rayyan—a web and mobile app for systematic reviews

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Synthesis of multiple randomized controlled trials (RCTs) in a systematic review can summarize the effects of individual outcomes and provide numerical answers about the effectiveness of interventions. Filtering of searches is time consuming, and no single method fulfills the principal requirements of speed with accuracy. Automation of systematic reviews is driven by a necessity to expedite the availability of current best evidence for policy and clinical decision-making.

          We developed Rayyan ( http://rayyan.qcri.org), a free web and mobile app, that helps expedite the initial screening of abstracts and titles using a process of semi-automation while incorporating a high level of usability. For the beta testing phase, we used two published Cochrane reviews in which included studies had been selected manually. Their searches, with 1030 records and 273 records, were uploaded to Rayyan. Different features of Rayyan were tested using these two reviews. We also conducted a survey of Rayyan’s users and collected feedback through a built-in feature.

          Results

          Pilot testing of Rayyan focused on usability, accuracy against manual methods, and the added value of the prediction feature. The “taster” review (273 records) allowed a quick overview of Rayyan for early comments on usability. The second review (1030 records) required several iterations to identify the previously identified 11 trials. The “suggestions” and “hints,” based on the “prediction model,” appeared as testing progressed beyond five included studies. Post rollout user experiences and a reflexive response by the developers enabled real-time modifications and improvements. The survey respondents reported 40% average time savings when using Rayyan compared to others tools, with 34% of the respondents reporting more than 50% time savings. In addition, around 75% of the respondents mentioned that screening and labeling studies as well as collaborating on reviews to be the two most important features of Rayyan.

          As of November 2016, Rayyan users exceed 2000 from over 60 countries conducting hundreds of reviews totaling more than 1.6M citations. Feedback from users, obtained mostly through the app web site and a recent survey, has highlighted the ease in exploration of searches, the time saved, and simplicity in sharing and comparing include-exclude decisions. The strongest features of the app, identified and reported in user feedback, were its ability to help in screening and collaboration as well as the time savings it affords to users.

          Conclusions

          Rayyan is responsive and intuitive in use with significant potential to lighten the load of reviewers.

          Related collections

          Most cited references6

          • Record: found
          • Abstract: found
          • Article: found
          Is Open Access

          Systematic review automation technologies

          Systematic reviews, a cornerstone of evidence-based medicine, are not produced quickly enough to support clinical practice. The cost of production, availability of the requisite expertise and timeliness are often quoted as major contributors for the delay. This detailed survey of the state of the art of information systems designed to support or automate individual tasks in the systematic review, and in particular systematic reviews of randomized controlled clinical trials, reveals trends that see the convergence of several parallel research projects. We surveyed literature describing informatics systems that support or automate the processes of systematic review or each of the tasks of the systematic review. Several projects focus on automating, simplifying and/or streamlining specific tasks of the systematic review. Some tasks are already fully automated while others are still largely manual. In this review, we describe each task and the effect that its automation would have on the entire systematic review process, summarize the existing information system support for each task, and highlight where further research is needed for realizing automation for the task. Integration of the systems that automate systematic review tasks may lead to a revised systematic review workflow. We envisage the optimized workflow will lead to system in which each systematic review is described as a computer program that automatically retrieves relevant trials, appraises them, extracts and synthesizes data, evaluates the risk of bias, performs meta-analysis calculations, and produces a report in real time.
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Systematic reviews and meta-analyses.

              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              A method for assessing the quality of a randomized control trial.

              A system has been constructed to evaluate the design, implementation, and analysis of randomized control trials (RCT). The degree of quadruple blinding (the randomization process, the physicians and patients as to therapy, and the physicians as to ongoing results) is considered to be the most important aspect of any trial. The analytic techniques are scored with the same emphasis as is placed on the control of bias in the planning and implementation of the studies. Description of the patient and treatment materials and the measurement of various controls of quality have less weight. An index of quality of a RCT is proposed with its pros and cons. If published papers were to approximate these principles, there would be a marked improvement in the quality of randomized control trials. Finally, a reasonable standard design and conduct of trials will facilitate the interpretation of those with conflicting results and help in making valid combinations of undersized trials.
                Bookmark

                Author and article information

                Contributors
                mouzzani@qf.org.qa
                hhammady@qf.org.qa
                zbysfedorowicz@gmail.com
                aelmagarmid@qf.org.qa
                Journal
                Syst Rev
                Syst Rev
                Systematic Reviews
                BioMed Central (London )
                2046-4053
                5 December 2016
                5 December 2016
                2016
                : 5
                : 210
                Affiliations
                [1 ]Qatar Computing Research Institute, HBKU, Doha, Qatar
                [2 ]Cochrane Bahrain, Awali, Bahrain
                Author information
                http://orcid.org/0000-0002-4035-3025
                Article
                384
                10.1186/s13643-016-0384-4
                5139140
                27919275
                ae2f0f9a-1405-44e4-9483-221a5734ea9e
                © The Author(s) 2016

                Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

                History
                : 21 January 2016
                : 16 November 2016
                Categories
                Methodology
                Custom metadata
                © The Author(s) 2016

                Public health
                systematic reviews,evidence-based medicine,automation
                Public health
                systematic reviews, evidence-based medicine, automation

                Comments

                Comment on this article