Scoping + Systematic Reviews

Data Extraction in Scoping vs Systematic Reviews

The Data Extraction Process

As with all of the other steps in the review process, best practice for data extraction is to have two independent reviewers extract data from each article. 

Beyond that, here are some tips for a smooth data extraction process.

1. Follow a Framework 
   1. For systematic reviews, this is typically the same PICO(T) framework you used to create your question.
   2. For scoping reviews, this is a bit trickier, since you often don't know what you're going to find until you find it. However, having a framework in mind can still help guide the extraction and analysis. For example, if your review involves social determinants of health, you might rely on the WHO's Social Determinants of Health. 
2. Create a Template
   Regardless of the type of review, a good template is essential. It helps everyone on your team know exactly what data they are looking for. Take a look at the section on data extraction templates in this guide.
3. Test Your Template
   Clarity is essential with a template. A template only works if everyone interprets its categories the same way. Even if your entire team creates the template together and seems to share an understanding about its categories, questions about interpretation may only become clear once data extraction begins. Therefore, you'll want to test it out with several articles and compare results across your team before jumping in. You don't want to get to the end of the extraction process just to realize you need to start over.
4. Standardize Your Language
   The more you and your teammates are alike in the data you extract, the easier it will be for you to move forward to data analysis. After you've tested your template, the conversation you have to compare results should include some discussion about the data you included in each of the template's categories to see if you can standardize that data as much as possible.

Data Extraction and Evidence Tables

Data extraction is the process of taking the most important characteristics of each study in the review and placing them into a chart, so that the details can be easily compared across studies. Typically, at least two people independently extract the data and then compare the results afterward, resolving any conflicts in the data that may have arisen in the process. Their data is then combined to create one final table.

While there are no prescribed set of study characteristics to include in a review, the most frequently extracted data items are things like:

• Authors and their affiliations
• Title
• Publication date
• Geographic location of the study
• Study design details
• Aims/objectives of the study
• Population studied
• Major findings
• Implications

There are many different ways to extract data. You can use a form that you build in Microsoft Forms, Google Forms, RedCap or in a tool like Covidence. (See the Review Tools and Applications tab.) You can enter the data directly into a spreadsheet program like Excel and then blend the charts later by copying and pasting. No matter how you extract the data, the important thing is to describe how you did it in the methods section and to reduce bias in the results by having the extractors work independently.

As mentioned earlier, you should also pilot your data extraction tool. To do this, have your extractors work on 2-3 articles to start. Compare answers and ensure that the results are similar. If your tool is lacking interrater reliability, it may be a sign that the extractors aren't understanding the data items the same way, and that definitions may need to be clarified.

More Advice

Examples of Evidence Tables

Data Extraction Templates