How to Write Systematic Review or Metaanalysis

Abstract

The Indian Journal of Orthopaedics (IJO) is the only orthopedic journal from India which is indexed with the Science Citation Index-expanded. Hence, it has an “Impact factor,” one of the most important journal matrix recognized by universities and academic institutions. Due to the constant hard work of the IJO team, the impact factor of our journal for 2017 increased from 0.79 to 0.98. This is the highest impact factor since the journal got indexed. Publication of well conducted systematic reviews and metaanalysis from wellknown researchers enhances the impact factor of a journal because they are frequently cited. This editorial discusses how one should plan and write a systematic review. Again as with the previous editorial on a similar theme,1 2 3 4 5 6 7 the aim is to help the authors prepare manuscripts which are readily acceptable. This is the era of evidence-based medicine. If one wants to practice evidence-based medicine, one has to integrate one's clinical expertise with the best available external evidence and patient's values and expectations. Systematic reviews and meta analysis of randomized controlled trials are considered the highest evidence in the hierarchy of evidence based medicine. Systematic reviews help to generate the best available external evidence by searching the literature systematically to answer a clinically relevant clearly defined review question.8 This is unlike narrative reviews which does not have a clearly defined review question, do not search the literature systematically and finally, do not present results and draw conclusions. The basic steps in conducting a systematic review are as follows: (i) defining the review question, (ii) defining the inclusion and exclusion criteria, (iii) systematic literature search, (iv) selection of the studies based on the previous inclusion and exclusion criteria, (v) assessment of the quality of the included studies, (vi) extraction of data from the included studies, (vii) summarizing the evidence, and (viii) discussing the review findings and drawing conclusion.9 Finally, the systematic review needs to be presented in a way that editors accept it for publication. Defining the review question: The first step in conducting a good systematic review is defining the problem to be addressed in the form of a clear, unambiguous, and structured question. One must initially choose an interesting broad area and then expand his knowledge about it by reading, discussing, and exploring. Once this is done, one must narrow down to a specific problem within the broad area [Table 1]. Finally, one must convert it into a review question [Table 1]. One of the ways which can be used to translate a clinical problem into a review question is to use the Patient/population, Intervention, Comparison, Outcome format (PICO) [Table 1]. In detail, the mnemonic refers to the following: Patients/Population: Defining the subject group like; age, sex, race, and other patient characteristics Intervention: Consider the intervention of interest Comparison: Group with whom the initially defined population and intervention would be compared to Outcome: The item you hope to accomplish, measure, or define Once the review question has been defined, usually no alterations should be made. One must ensure that the question that is formulated is clinically relevant, novel, and interesting. Defining the inclusion and exclusion criteria: When conducting any systematic review, it is very essential that the authors explicitly define the studies which they would select and those they would exclude. Usually, high evidence level studies such as randomized control trials (RCTs) and well-conducted level II studies must be included. However, if the review question is such that not much work has been done in that field then it is prudent to include lower level studies also so that some useful conclusions can be derived at the end of the study. It is also important to decide the language whose article would be included. Although including only English language articles does introduce a bias in the systematic review, it may be essential if translation facilities and other resources are sparse. It is also a good idea to define the time frame of publication of the included studies. It should also be defined if only human studies would be included or both human and animal studies would be included Systematic literature search: The literature search strategy must be defined. One must identify the main themes within the review question and find as many keywords/Medical Subject Headings (MeSH) terms for each theme. Following this, the keywords need to be connected using appropriate Boolean operators [Table 1]. All possible relevant electronic data basis must be searched. For the cochrane review a minimum of three databases need to be search. Some of the data basis which should be included are PubMed, MEDLINE through PubMed, Embase, Web of Science, Scopus, and Cochrane Controlled Trials Register. Appropriate search filters such as duration, type of studies (animal or human), language etc. must be used at this stage. The studies thus identified must be exported to an efficient reference manager software such as Mendeley, Zotero, EndNote etc. Besides electronic database search, a hand search of lankmark articles published in that field must be done. A search of the “grey literature” must be done to include additional studies. This includes materials and research produced by organizations outside the traditional commercial or academic publishing and distribution channels such as reports, working papers, government documents, white papers etc. Selection of the studies based on the previous inclusion and exclusion criteria: Once all relevant articles have been collected and duplicates removed, the title of each article must be read to removed the irrelevant ones. This should usually be done by two reviewers. Any article where consensus has not been reached should be retained, at this stage. The abstracts of all the remaining articles must then be read to eliminate further articles. Articles must also be removed if they do not meet the inclusion criteria. Full text of all articles which are left after this must be read to decide if they should be included or not. The references of the included articles must be checked to identify other relevant studies that can be included again based on the original inclusion criteria. A Preferred Reporting Items in Systematic Reviews and Meta-analysis (PRISMA) flow chart depicting the exact flow and number of studies must be included [Figure 1] The assessment of the quality of the included studies: Once the studies that are shortlisted for final inclusion in the systematic review have been identified, their quality analysis must be done. The design and level of evidence of the included studies must be ascertained. The internal and external validity of included studies must be assessed. Any bias such as description bias, selection bias, measurement bias, analytic bias, and interpretation bias must be assessed. Available quality scales or checklists like critical appraisal skills program (CASP) checklists can be useful tools, but their strengths and weaknesses must be known and described Extraction of data from the included studies: Once the quality of the articles has been ascertained the relevant data from every study, based on the review question needs to be extracted. The data that is extracted should be meticulously filled in a well designed spreadsheet. Initially as much data as possible should be extracted so that anything important is not missed because that would require going through all the manuscripts again. Summarizing the evidence: The extracted data need to be summarized to draw valid and logical conclusions. When the data extracted for the interventions and outcomes being studied in the review question, is similar enough that it can be pooled together using statistical tools, then it should be done to produce a quantitative review which is called a meta analysis.10 Review Manager (Rev Man 5) is a very useful tool for preparing and maintaining cochrane reviews.11 If that is not possible then a qualitative review should be produced. When a qualitative summation is done each outcome that has been mentioned must be reported separately and hence named systematic review Discussing the review findings and drawing conclusion: The discussion should include the key findings about each of the main outcomes. The strength of evidence about each outcome measure must also be discussed. The limitations and strengths of the included studies and the authors own reviews must also be included. The results obtained must be compared to those of the other studies. How the results can affect clinical practice, policy and future research must be discussed. Sometimes, the results may be inconclusive, but even then they should be reported because it enlightens researchers to conduct research in that area. Table 1 Formulating a review question and search strategy Figure 1 PRISMA flow chart For wide dissemination of the results of a systematic review, it is imperative that it is published in a journal of repute. For this, it is not only important that the review is conducted properly, but also well presented. The preferred Reporting Items in Systematic Reviews and Metaanalysis (PRISMA) checklist readily available on the website www.equator.net, is the most authentic and commonly used checklist, which all authors must consult.12 All systematic reviews and meta analysis are to be registered with PROSPERO (www.crd.york.ac.uk/prospero/) and registration number provided at the time of submission of manuscript. Conducting a systematic review, writing it up and finally publishing it, is a mammoth task. Authors who are willing to take up the challenge must equip and prepare themselves for the task. The final fruit of the effort which is getting ones well-conducted systematic review published in a peer review, indexed journal is worth all the efforts.