Just in case it's been a while since your last research paper, this page is a refresher on how to think about researching a topic for a literature review or other assignment. Included is a strategic process to guide your searching and keep you organized.
Finding Empirical Research
For many of your assignments, you will be asked to find empirical research studies, also called original research. Empirical studies are papers about observational or experimental research. They are different from articles like review articles, which discuss several different studies but don't report new experimental results. Empirical research is considered to be a primary source while review articles are secondary sources. (See the box below for more on primary sources.)
How can you tell if the article you found is empirical research? Look at the sections of the article. Most empirical studies will have the following five sections:
Empirical research articles are published in scholarly, peer-reviewed journals and can be found in the library's databases, as well as in publicly accessible databases like PubMed.
Empirical is not a study design, but a term for any study that is experimental or observational. Empirical research can be quantitative, qualitative or mixed methods. Randomized controlled trials, cohort studies, and ethnographic studies are all examples of empirical research.
In the health and social sciences, most often a primary source is a report of the results of an experiment or research study. The most common source format these take are published articles found in scholarly journals or conference papers.
Why Choose Primary Sources
Primary Sources represent the best evidence to support an argument. When you examine a primary source, you get to see all of the relevant information about the study as well as the authors' interpretation of the results of that study. You can then come to your own conclusions about the significance, relevance, and meaning of those results as they apply to your topics of interest. If instead you rely on the interpretations of another author, say from a literature review or other secondary source, you can run into trouble, because their point of view might cloud their interpretations and could lead you astray. Therefore, when you can, you will want to select primary sources to cite in your papers and other academic work.
Identifying Primary Sources
However, not everything published in a scholarly journal or presented at a conference will be a primary source. Here are some things to look for.
PICO Questions and Search Strategies
Think of a search strategy as a shopping list for evidence. When you're looking for evidence, there is probably specific types of evidence fitting specific criteria that you need. Planning your strategy in advance can save you some time and keep you organized. Furthermore, having a record of what, where and how you searched can help you replicate your search later or make modifications to improve your results.
Before you start a topic search for a literature review or other assignment, ask yourself these questions:
Placing your research question in the PICO(T) format can help you ensure that your question is answerable. If your question doesn't fit PICO(T), it might be too specific (not enough results) or too general (too many results) to research using the databases. The video tutorial below will take you through the process of developing a PICO(T) question.
This PICO(T) Templates worksheet can also help.
A PICO(T) question also makes searching easier because the type of question determines what type of evidence you should look for:
| Type of PICO(T) Question | Type of Evidence Needed to Answer |
| Diagnosis | Studies reporting performance characteristics |
| Therapy | Clinical trials Randomized Controlled Trials Systematic Reviews (of Clinical Trials and RCTs) |
| Etiology | Cohort Studies Case Control Studies Systematic reviews (of Cohort and Case Control Studies) |
| Prognosis | Cohort Studies |
| Meaning | Qualitative Studies: Ethnography, Phenomenology, Grounded theory, Focus groups |
Remember that some types of evidence are stronger than others. Often evidence is shown in a pyramid, in which each ascending level represents a different type of study design and corresponds to increasing rigor, quality, and reliability of the evidence. In other words, as we ascend through these different study designs, we become more confident that their results are accurate, have less chance of statistical error, and minimize bias from confounding variables that could have influenced the results. (http://www.students4bestevidence.net/the-evidence-based-medicine-pyramid/)
Always aim for the highest level of evidence appropriate to your question type. For your DNS 820 lit review, you will probably be asked to use original research studies only and not systematic reviews. However, if you find a systematic review that answers your question, you can use it to lead you to original research studies relevant to your question.
Keywords and Subject Headings
Keywords
When you search with keywords, you are trying to find words that authors have used in their titles and abstracts or somehow otherwise appear in the article's entry in a database. Therefore you need to brainstorm all of the possible ways authors could refer to your concept:
"endurance" OR "physical fitness"
"education" OR "training" OR "school" OR "learning"
Subject Headings
You can also search using subject headings. Subject headings are keywords assigned by the databases to describe the concepts in an article and to try to take some of the guesswork out of the job of coming up with keywords. Try searching with subject headings and see how your results differ from searching with just keywords.
Still unsure about subject headings? Watch this video for another explanation as well as tips on how to find them.
OneSearch isn't always the best search tool for a literature review. The individual databases like PubMed, CINAHL and others have additional features that OneSearch does not that make searching easier. Furthermore, the databases search different content, and one database may be more applicable to your topic than another. When preparing to search, decide which databases will have the content that you need. For example, if your question is about nursing, you'll definitely want to search CINAHL. If your question is about mental health, you'll definitely want to search PsycInfo. For more about the individual databases, including video tutorials that will teach you how to search them, please visit the tutorial pages below.
Once you have some search results, you will need to decide which articles you will actually use in your literature review. This can be done using filters/limits in the databases, applying inclusion/exclusion criteria, and appraising the articles.
The databases can help you narrow down your search results by criteria such as publication year, language, age group, and publication type. This feature, which is called Filters or Limits, usually appears to the left of the search results, such as in PubMed and CINAHL. In Ovid/PsycInfo, the limits option is located above the search results. (See the Searching with Library Tools tab for tutorials.)
Usually, filters/limits are not enough criteria to narrow down your results to only the best, most essential evidence. The final step in deciding which articles to review is to apply Inclusion Criteria. If a study meets your inclusion criteria, then it would be an appropriate piece of evidence to help answer your PICO(T) question. If a study meets one or more of your exclusion criteria, it will not help answer your PICO(T) question and you can discard it. You will have to read titles and abstracts to determine if articles meet your inclusion criteria.
Using criteria like these examples will help you decide if a study will answer your PICO(T) question.
PICO(T): Do the patient population, intervention, comparison and outcome match the ones in your clinical question? If one of these is different in the study from your question, you will need to exclude it.
Age: Are you working with a specific age group? The study should include subjects that are within your specified age range.
Setting: Are you looking for studies in a specific setting? i.e. home, outpatient, inpatient, etc.
Health Status: For example, this could be patients with good or poor prognosis.
Study Design Preference: Are you looking for RCTs only? Will you also include other types of trials or cohort studies?
Size of Study Groups: What is the minimum study group size you are willing to consider?
Study Drop Out Rate: Are you looking for studies with drop out rates <20%? <10%?
Year Range: Studies published within the last 5 years? 10 years?
Language: Studies published in English? Other languages?
|
Exclusion criteria is the inverse of your inclusion criteria. For instance, if your inclusion criteria for study drop out rate is <20%, the exclusion criteria would be a drop out rate of >20%. |
When considering studies for inclusion, ask yourself, is this relevant to my patient or the problem?
Once you begin reading an article, you may find that the study population isn't representative of the patient or problem you are treating or addressing. Research abstracts alone do not always make this apparent.
You may also find that while a study population or problem matches that of your patient, the study did not focus on an aspect of the problem you are interested in. E.g. You may find that a study looks at oral administration of an antibiotic before a surgical procedure, but doesn't address the timing of the administration of the antibiotic.
Once you have chosen the studies that have potential to answer your question, you can then evaluate them for things like validity, reliability and quality.
Adapted from Evidence-Based Practice for Nursing: Evaluating the Evidence and Evidence Analysis Manual
What to Look for When Reading an Article
By this point, you should have a narrowed your articles for inclusion in your literature review to a reasonable number. Now it's time to read the articles you have selected and decide if they are high quality evidence. Here are questions to ask yourself while reading an article that will help you make the final decisions about which articles in your results are truly the best evidence.
Introduction
Should provide a proper background for the study.
What is the study purpose/aim/objective? Is there a research hypothesis? Were these answered in the results and discussion
Materials and Methods
Were the procedures and equipment used for data collection appropriate?
Results
Were the results reliable and valid?
Did any subjects drop out of the study? If yes, is an explanation given?
Determine which results are statistically significant and which are not.
Were appropriate statistical tests used?
Are statistically significant results also clinically significant?
Discussion
What was the final answer to the research question?
How did the authors interpret the results and their analyses?
Did this study lead to new theories or hypotheses, and suggestions for future research?
What did the authors identify as the shortcomings or limitations of the study?
Did the authors compare/contrast their study with similar studies?
Note, the discussion of results is ultimately an opinion and not automatically fact.
Conclusion (Again)
Were the inferences that you made at the beginning correct?
If the conclusion didn’t make sense at first, does it make more sense after having read the entire article?
Adapted from Subramanyam, R. (2013). Art of reading a journal article: Methodically and effectively. Journal of Oral and Maxillofacial Pathology : JOMFP, 17(1), 65–70. http://doi.org/10.4103/0973-029X.110733
Modifying an Unsuccessful Search
If you aren't happy with the list of articles your search brings back, here are some ways you might think about changing your search.
1. Too many articles / Articles aren't on topic
2. Too few articles
Photo by Anant Nath Sharma, used with permission under a Creative Commons license
