Systematic Reviews & Evidence Synthesis

• Systematic search design is based on structured “concept” searching
• Comprehensive searching is as sensitive as possible but as precise as necessary. To achieve this, these search strategies:
  • Always uses both keywords AND controlled vocabulary, when available
  • Uses the database syntax and keyword modifications (field tags, Boolean, modifiers, proximity operators) to balance sensitivity and precision
  • Relies on database searching for structure and additional methods to extend the comprehensiveness
• Systematic searches should always be peer reviewed
• Search design and process should be transparently reported

Systematic and comprehensive searches are based on structured concept searches.

Using a research question framework can help start this process. Select a framework based on your review type (or, for a structured review, the most similar review type).

Below are two common frameworks as examples, but there are so many more! Check the guide to 'Using a framework to structure your research question' from the University of Plymouth Libraries for more examples of structured frameworks for evidence synthesis research questions.

These framework components form the core structure of your search. You will connect multiple search terms for each concept into a structure similar to:

(PopulationTerm OR PopulationTerm) AND (InterventionTerm OR InterventionTerm) AND (OutcomeTerm)

As you plan this, be wary of:

• One framework component (population, intervention, etc.) including multiple, unique search concepts (e.g. (PopulationTerm OR PopulationTerm) AND (PopulationTerm OR PopulationTerm))
• One search concept appears in multiple framework components and can be combined and added only once to the search design
• You (usually) will not include all of your framework concepts in your search design. Often you focus the search on population and intervention/exposure/concept.

Select (usually) 3-8 databases total. These include:

• Primary databases for your topic/field
  • Select one of these for your “main search design" database
  • Select this database based on coverage and inclusion of subject headings
• General databases that include that topic/field (Google Scholar does not count)

Potential Main Search Design Databases

• PubMed (Medline) is the most common main search design database for health, biomedical, veterinary medicine, public health and many other review topics. You can alternatively use Medline via EBSCO which is almost exactly the same content. Select this option if you will be searching many databases via EBSCO.
• ERIC is a common main search design database for education topics.
• CINAHL is a common main search design database for occupational therapy or any allied health topics.
• PsycInfo is a common main search design database for psychology or if the topical focus is on behavior or mental health.
• CAB Abstracts is a common main search design database for animal sciences, veterinary medicine, and natural resources.
• Business Source Complete is a common main search design database for business and leadership topics.

There are many more to pick from. Ask for librarian assistance if you are not sure which to select.

Select Other Databases

To identify other databases most relevant to your topic/field, visit the CSU Libraries A-Z Database List, then use the limiting filters above the list to limit to select a specific subject area, e.g. Natural Resources or Psychology.

Select subject lists based on not just your field of research, but any fields that might overlap.

For instance, if your research project is on effective marketing interventions for increasing adoption of energy-efficient home appliances, then you may want to limit the A-Z Database List to subjects like Sustainability, Business Marketing, and maybe even Engineering or Journalism & Media Communication. Look for overlapping databases as well as major databases in each subject area.

Documentation is essential BOTH during search design and when running the final searches. Make sure you are familiar with the reporting guidelines for search designs and consider using a template to help you record all necessary information.

Documentation Templates & Guidelines

During Search Design

You need to clearly report not just the terms you used, but the full search structure exactly as entered in each database for transparent reporting in the final manuscript.

Start documenting this early by having a place, such as the template linked above, where you document your search design separately for each database you use.

Good documentation of your search design process will save you time by helping you know clearly which terms, databases, and search structures you have already tested. Keep track of terms you test as well as any time you test different field tags, wildcards, keyword modifiers, proximity operators and other special search syntax described below.

Raw Data

For Evidence Synthesis projects, an important type of 'raw data' are the export files you get when you download your results out of a database.

These are often .ris files, but they may be .nbib, .txt or other types of files. These are the files you can load into a citation manager, a duplicate removal tool, a screening tool, etc

CLEARLY LABEL AND SAVE UNALTERED VERSION OF THESE FILES!

Label these files with:

1. Shortened project name
2. Name of the database
3. Name of the platform
4. Date the search was run
5. Total number of results that should be in the file

It might look something like:

• REGSR_PsycInfo_Ebsco_13Feb2024_678

Term Types

There are two types of essential search terms: keywords and subject headings (also known as controlled vocabulary).

For each of your search concepts, you need to build a comprehensive list that includes both types of terms.

For subject headings, start in your main search design database first. Gather equivalent subject headings from other databases later. Always keep track of which database a subject heading is used in - they are unique to that database in most cases.

Use Seed Articles to Harvest Search Terms

Starting in your main search design database, look up the record for each seed article you have found. Check the record's title, abstract, author supplied keywords, and subject headings.

Remember, you don't need every subject heading on the article, just the ones that directly relate to a concept in your own search design.

Brainstorm or Make a Concept Map

Relying on seed articles alone can lead to bias (under or over representation of a term) in your search.

As you do exploratory searching, keep a list of keywords you notice and brainstorm or make a concept map to identify additional terms to try.

Remember, for evidence synthesis projects, your balance of precision to sensitivity will always favor sensitivity! If it COULD add a relevant result, then you SHOULD add the term to the search.

If the results are too big to manage with your project timeline and resources, contact a librarian for assistance with appropriate search techniques.

Use a Tool for Term Analysis

There are a variety of online tools for term analysis. Some of the best are specifically meant for PubMed and look at both keywords and subject headings. Other tools just focus on keywords.

Modifying Keyword to Enhance Your Search

Use these common modifiers to enhance your keywords.

Truncation

Rather than repeatedly including one term with many word endings, put a star * after the word root.

Concept* = concept, concepts, conception

Phrase Searching

If you want to search for a phrase, make sure to add quotations around it. If you don't add quotation marks, the database does not know that those two words should be connected to each other.

"Quotation marks" = only that exact set of words in that exact order. You will not retrieve 'marks of quotation' or 'quotation mark'

Wildcards

Many databases have 'wildcards' which are symbols that represent the potential addition of 0, 1, 2, or 3 letters in the middle of a word. This is especially useful for UK and US spelling.

These symbols are database specific! PubMed does not have wildcards, but all EBSCO databases and Web of Science do.

EBSCO

P#ediatric = pediatric or paediatric

Web of Science

P$ediatric = pediatric or paediatric

Special Considerations for Subject Headings

Unlike a keyword, you should never alter or modify a subject heading!

1. Subject headings are unique to each individual database
2. Do not change the order or terms, add or remove commas, change plural to singular (or vice versa), or use truncation (the *)
3. On a PubMed article record, the * after a MeSH term on an article record indicates it is the major focus of the article. It is NOT part of the MeSH term and should be left off.
4. On a PubMed article record, the structure "Term / Term" indicated a MeSH term with a subheading. Subheadings are the second part after the slash, and they make a MeSH term more specific. Generally, just use the MeSH term before the / in a comprehensive search design.

What is '"Database Syntax"?

Database syntax refers to the specific structures and codes that a database uses to form search queries in a systematic manner. These can be both universal (search syntax) and database specific (database syntax). Core aspects of syntax are: field tags, Boolean operators, and proximity operators.

When working on database syntax, always check the "Help" documentation for the database you are using!!

Field Tags

Field tags are always used in systematic searches to increase reproducibility and transparency.

Field tags tell the database to search for that term in only a specific field of the article records. That field could be the title of the article, the abstract, author supplied keywords, the subject headings, etc.

Keyword Field Tags

Subject Heading Field Tags

*The large variety of databases on the EBSCO platform means that there may be variations specific to the database(s) you are using. Contact CSU Libraries if you want to confirm that you are using the proper field tags for the specific databases you are searching.

++In most cases, terms tagged with DE and enclosed in quotation marks are searched as both subject headings and as author supplied keywords

Boolean Operators

Boolean operators are a universal type of database syntax used to group together terms and define the order of operations for a search.

AND = combine two different concepts; all results must include both concepts

OR = combine alternative terms for the same concept; all results must include at least one of those terms

Do not use NOT in systematic search design except with a validated search hedge.

Parenthesis group terms together so it is clear to the database, which 'calculations' are run in which order.

As syntax this looks like:

(PopulationTermA OR PopulationTermB OR PopulationTermC) AND (InterventionTermX OR InterventionTermY OR InterventionTermZ)

Proximity Operators

Some databases have operators that connect terms loosely by being close to each other but not necessarily in a perfect phrase structure. If "quotation marks" make a phrase highly precise, proximity operators allow for a little more natural variation in how those words are connected.

Only use these with keywords, not subject headings.

Searching using additional methods beyond database searches enhances the comprehensiveness of the project.

In order to maintain the structure and systematic process, plan these methods in the protocol phase, identify methods that are as reproducible as possible, and transparently report all steps taken.

Generally, some form of citation searching of the references and citing articles for all included studies (or the seed articles) is the minimum expectation for 'additional methods'. Select other methods based on the specifics of the literature and research process in the topic and field(s) where you are searching.

Where Else Do I Search?

References (backward in time)

Check all references from the papers you include in your final analysis against your selection criteria. If you have a large and well curated set of seed articles that did make it through screening, then this process is sometimes done only on the seed articles and not on all included articles. Tools like Web of Science or Citationchaser can increase efficiency.

Cited By (forward in time)

Use Google Scholar, Web of Science or special tool like Citationchaser on all articles you selected for analysis to check for articles that cited them since they were published. Check all these articles against your selection criteria. Again, this process is sometimes done only on the seed articles and not on all included articles.

Citation Searching (backward & forward in time)

Do both! Web of Science and Citationchaser are better tools than Google Scholar if you check citations on included articles or seed articles both backwards  and forwards in time.

Generally these citations (either backwards and/or forwards in time) are screened using the same criteria and often the same process as the database search results. HOWEVER,the PRISMA Flow Diagram records databases searches separately from additional method, so make a fresh project for this process in your screening tool or use a specific 'additional methods' screening feature if the tool offers it so that you retain accurate counts.

"Hand" Search Specific Journals

Select a set of journals that publish regularly on the topic of interest in your review. This may be done in consultation with research advisors, knowledgeable colleagues or consultation with a librarian. Use BrowZine, journal websites or databases to scan the titles and abstracts of all issues of that journal (either in a selected time frame or as far back as possible).

Alternatively, use a database that indexes the selected journal to export all the articles from that journal and import them into a screening tool. Screening may be done exactly the same as regular database search results or you may modify it to be three stages - title screening, then abstract screening, then full text screening - as this can make large, diverse results sets faster to screen, especially with a large enough research team.

Dissertations and Theses

Many valuable sources of evidence are published as dissertations or theses. They are also often working in newer and emerging areas of study. Try running your search design through ProQuest Dissertation and Theses Global (it will need to be translated to the proper database syntax). Or try DART-Europe E-theses Portal or Networked Digital Library of Dissertations & Theses (NDLT) for more international coverage. Some dissertation and theses indexing tools are treated just like database searches, and some are treated like 'additional methods'. Ask a librarian for clarification.

Clinical Trial Registries

Clinical trial registries collect the registrations of the intention and research plan for conducting a clinical trial. Not all of these end up being published, even if they produced results. It can be valuable to find records of clinical trials that were planned on your topic to see if you collected the published result or potentially contact authors to see if they are able to provide you with results for your project.

Clinical trial searches are generally conducted and screened along with regular database searches. In the PRISMA Flow Diagram, the result numbers are recorded in the same box as the database search numbers, but they recorded on a separate line.

Google Scholar

You can just design and run a search in Google Scholar. Google Scholar does not have field tags and the search syntax is limited, so results tend to be large. It is also difficult to export results from a search. Generally, in evidence synthesis projects, you test a search in Google Scholar and review the results, decide the how many results from Google Scholar you will screen, then use a tool like Publish or Perish to run and export the search and number of results you selected.

Grey Literature

Grey literature is a term for publications that do go through the scholarly publication process (peer review, particularly). Large amounts of research and other data are produced and published outside scholarly channels, so try one or more relevant databases, like Mednar or OAIster.

Preprints