Literature Reviews: Systematic, Scoping, Integrative

Elements of a Systematic Review

This section of the guide will not address every element of formal literature reviews but some of those that are unique to this type of research and those that seem to engender the most questions.

Methods Section 

The methods section of any formal review should include enough detail that a reader could reproduce the search faithfully and follow the same process you used to review articles for inclusion. Here is a list of elements it should include.

• Sources of articles, etc.
  • Names of the databases you searched, including the database platform and the coverage dates
  • Whether you will be mining reference lists for additional sources
  • Grey literature sources, including how you identified them and whether there were differences in the search techniques
• Your search terms, differentiating between keywords and subject headings. You can make an exception to this if you include your search terms in an appendix, in which case you might provide a summary of your terms in the body of the methods section (see the Search Strategy Table section for more details). At this point, you would also cite any methods you used to review your search strategy.
• The date you conducted the search. A month, day, and year is ideal.
• Filters or limits you applied to the search (e.g. limited to English language articles).
• Inclusion/exclusion criteria

Results

The results section provides details about your search results, your process for reviewing those results, and the evidence you extracted from the results. It will include the following elements.

• Number of articles, etc. retrieved. Report those numbers separately based on the source: databases (including number of duplicates removed), clinical trial registries, reference lists, and grey literature. You may not have articles from all of these sources, depending on the scope of your review.
• The number of articles you excluded during each round of screening based on your inclusion/exclusion criteria.
• PRISMA Flow Diagram.
• The methods you used to review titles/abstracts and full-text 
  • Did you use Covidence or another application?
  • How many people reviewed each source?
• Data extraction (additional details below) 
  • How many people were involved
  • What method did you use (Covidence, spreadsheet, etc.)
  • Summary list of the data points you extracted
• Data or evidence table, displaying the data you extracted.

PRISMA Flow Diagram

A PRISMA Diagram is a flow chart that shows the decision making path you took to eliminate search results and land on a final group of information sources to review. The content can vary based on the type of review and the types of sources you choose to include. Here is an example

For pdf or Word templates, visit the PRISMA website.

PRISMA flow diagrams can also be generated using a Shiny App available at https://www.eshackathon.org/software/PRISMA2020.html

For more information, see Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J.M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S.,...Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Medicine, 18(3), Article e1003583. https://doi.org/10.1371/journal.pmed.1003583 

Data Extraction and Evidence Tables

Data extraction is the process of taking the most important characteristics of each study in the review and placing them into a chart, so that the details can be easily compared across studies. Typically, at least two people independently extract the data and then compare the results afterward, resolving any conflicts in the data that may have arisen in the process. Their data is them combined to create one final table. 

While there are no prescribed set of study characteristics to include in a review, the most frequently extracted data items are things like:

• Authors and their affiliations
• Title
• Publication date
• Geographic location of the study
• Study design details
• Aims/objectives of the study
• Population studied
• Major findings
• Implications

There are many different ways to extract data. You can use a form that you build in Microsoft Forms, Google Forms, RedCap or in a tool like Covidence. (See the Review Tools and Applications tab.) You can enter the data directly into a spreadsheet program like Excel and then blend the charts later by copying and pasting. No matter how you extract the data, the important thing is to describe how you did it in the methods section, and to reduce bias in the results by having the extractors work independently.

You may also want to pilot your data extraction tool. To do this, have your extractors work on 2-3 articles to start. Compare answers and ensure that the results are similar. If your tool is lacking interrater reliability, it may be a sign that the extractors aren't understanding the data items the same way, and that definitions may need to be clarified.

Examples of Evidence Tables

Search Strategy Table

Depending on the complexity of your search and how many terms it contains, you may not want to include all of the details in the text of the Methods section of your paper. In that case you might opt for an appendix or other supplemental material, depending on what the journal you're submitting to allows.

There are several ways you might format the search strategy. In Cochrane reviews, the terms are simply laid out in a list for each database searched (see Appendix 1 in this Cochrane review). 

You could also opt for a table that implies the relationship between the search terms, like this one: