Essential Features of Comprehensive & Systematic Searches
- Systematic search design is based on structured “concept” searching
- Comprehensive searching is as sensitive as possible but as precise as necessary
- Always uses both keywords AND controlled vocabulary, when available
- Uses the database syntax (field tags, Boolean, modifiers, proximity operators) to balance sensitivity and precision
- Relies on database searching for structure and additional methods for comprehensiveness
- Systematic searches should always be peer reviewed
- Search design and process should be transparently reported
- Evidence Synthesis Search Design Documentation - TemplateThis template can be used to document both decision making and final reporting requirements for an evidence synthesis search design.
- Develop a PubMed Search Strategy for a Systematic Review - Health Sciences Library, UC San DiegoThis is an excellent video (27 min) on putting together a systematic and comprehensive search design. It uses PubMed as the example database.
- Sample Search DesignThis example search is done in an EBSCO database (on a social work topic) and includes examples of all the features described below. You can use it in conjunction with the Search Design Documentation Template and materials below to develop your own search strategy. If you need assistance with this process or help translating it for use in PubMed, please contact me.
Research Question Concept Frameworks
Systematic and comprehensive searches are based on structured concept searches. Using a research question framework can help start this process. Select a framework based on your review type (or, for a structured review, the most similar review type).
- Using a framework to structure your research question - University of Plymouth LibrariesAdditional research question frameworks and details
Focused Questions - PICO (systematic review) | Broad Questions - PCC (scoping review) |
---|---|
Population = population characteristics, condition/problem, patient situation Intervention = what is occurring to the 'population' Comparison = alternative intervention or control Outcome = measurable (clinical) outcomes of interest |
Population = population characteristics, condition/problem, patient situation Concept = the core concept of the review. May be interventions and/or phenomena of interest and/or outcomes. Often there are multiple concepts. Context = setting of study, geographic location, specific racial or gender-based interests |
Sometimes there is no comparison (PIO) Interventions may be: a treatment, exposure, diagnostic test, prognostic factor, patient perception, etc Add Time = duration of treatment, length of time to measurement Add Type = the evidence type (e.g. study/research design like Controlled Trial or RCT) Add Setting = setting of the study or applicable real word situation |
There is often multiple Concepts or Contexts, and sometimes no Population or Context. This framework is applied flexibly, but all parts must ultimately be defined clearly. Add Study Type = types of study/research design/evidence type |
These framework concepts form the core structure of your search. You will ultimately connect multiple search terms for each concept into a structure similar to:
(PopulationTerm OR PopulationTerm) AND (InterventionTerm OR InterventionTerm) AND (OutcomeTerm)
As you plan this, be wary of:
- One framework component (population, intervention, etc.) including multiple, unique search concepts
- One search concept appears in multiple framework components
- You (usually) will not include ALL of your framework concepts in your search design. Often you focus the search on population and intervention/exposure/concept.
Select Databases
Select (usually) 3-8 databases total. These include:
- Primary databases for your topic/field
- Select one of these for your “main search design" database
- Select this database based on coverage and inclusion of subject headings
- General databases that include that topic/field (Google Scholar does not count)
Potential Primary Search Design Databases
- PubMed (Medline) is the most common main search design database for health, biomedical, veterinary medicine, public health and many other review topics. You can alternatively use Medline via EBSCO which is almost exactly the same content. Select this option if you will be searching many databases via EBSCO.
- ERIC is a common main search design database for education topics.
- CINAHL is a common main search design database for occupational therapy or any allied health topics.
- PsycInfo is a common main search design database for psychology or if the topical focus is on behavior or mental health.
- CAB Abstracts is a common main search design database for animal sciences, veterinary medicine, and natural resources.
- Business Source Complete is a common main search design database for business and leadership topics.
There are many more to pick from. Ask for librarian assistance if you are not sure which to select.
Harvesting Terms
Term Types
There are two types of essential search terms: keywords and subject headings (also known as controlled vocabulary).
Keywords | Subject Headings |
---|---|
Language used, without standardization, to describe concepts
|
One word/phrase tagged to all articles about that concept (in that database)
|
For each of your search concepts, you need to build a comprehensive list that includes both types of terms.
For subject headings, start in your primary search design database first. Gather equivalent subject headings from other databases later. Always keep track of which database a subject heading is used in - they are unique to that database in most cases.
Use Seed Articles to Harvest Search Terms
Starting in your primary search design database, look up the record for each seed article you have found. Check the record's title, abstract, author supplied keywords, and subject headings.
Remember, you don't need every subject heading on the article, just the ones that directly relate to a concept in your own search design.
- Worksheet - Seed Article Term HarvestingUse this worksheet to help you harvest keywords and subject headings from your seed articles for use in your search design.
Brainstorm or Make a Concept Map
Relying on seed articles alone can lead to bias (under or over representation of a term) in your search. As you do exploratory searching, keep a list of keywords you notice and brainstorm or make a concept map to identify additional terms to try.
Use a Tool for Term Analysis
There are a variety of online tools for term analysis. Some of the best are specifically meant for PubMed and look at both keywords and subject headings. Other tools just focus on keywords.
- Yale MeSH AnalyzerEnter a list of PMIDs from seed articles and get back an alphabetized list of medical subject headings (MeSH) for each article.
- PubMed PubReMinerRun a search strategy or a list of PMIDs (separated by commas) through the search to get an analysis of words (keywords), MeSH and other features of the articles.
- Word Freq - Systematic Review AcceleratorUpload a collection of article records from a citation manager or database in .xml, .ris, or .nbib format and get a word frequency analysis report.
Modifying Keyword to Enhance Your Search
Use these common modifiers to enhance your keywords.
Truncation
Rather than repeatedly including one term with many word endings, put a star * after the word root.
Concept* = concept, concepts, conception
Phrase Searching
If you want to search for a phrase, make sure to add quotations around it. If you don't add quotation marks, the database does not know that those two words should be connected to each other.
"Quotation marks" = only that exact set of words in that exact order. You will not retrieve 'marks of quotation' or 'quotation mark'
Wildcards
Many databases have 'wildcards' which are symbols that represent the potential addition of 0, 1, 2, or 3 letters in the middle of a word. This is especially useful for UK and US spelling.
These symbols are database specific! PubMed does not have wildcards, but all EBSCO databases and Web of Science do.
EBSCO
P#ediatric = pediatric or paediatric
Web of Science
P$ediatric = pediatric or paediatric
Special Considerations for Subject Headings
Unlike a keyword, you should never alter or modify a subject heading!
- Subject headings are unique to each individual database
- Do not change the order or terms, add or remove commas, change plural to singular (or vice versa), or use truncation (the *)
- On a PubMed article record, the * after a MeSH term on an article record indicates it is the major focus of the article. It is NOT part of the MeSH term and should be left off.
- On a PubMed article record, the structure "Term / Term" indicated a MeSH term with a subheading. Subheadings are the second part after the slash, and they make a MeSH term more specific. Generally, just use the MeSH term before the / in a comprehensive search design.
Database Syntax
What is '"Database Syntax"?
Database syntax refers to the specific structures and codes that a database uses to form search queries in a systematic manner. These can be both universal (search syntax) and database specific (database syntax). Core aspects of syntax are: field tags, Boolean operators, and proximity operators.
When working on database syntax, always check the "Help" documentation for the database you are using!!
Field Tags
Field tags are always used in systematic searches to increase reproducibility and transparency.
Field tags tell the database to search for that term in only a specific field of the article records. That field could be the title of the article, the abstract, author supplied keywords, the subject headings, etc.
Keyword Field Tags
Database | Article Title | Abstract | Author Supplied Keywords | Title, Abstract, and Keywords | Structure of Multiple Keywords in Multiple Fields for One Concept |
---|---|---|---|---|---|
EBSCO | TI Keyword | AB Keyword |
KW Keyword
|
TI Term OR AB Term OR KW Term |
(TI (KeywordA OR KeywordB) OR AB (KeywordA OR KeywordB) OR KW (KeywordA OR KeywordB) ) |
PubMed | Keyword [TI] | Keyword [AB] | Keyword [OT] | Keyword [TIAB] | (KeywordA [TIAB] OR KeywordB [TIAB]) |
Web of Science | TI=Keyword | AB=Keyword | AK=Keyword |
TS=Keyword *Also includes Keyword Plus |
TS=(KeywordA OR KeywordB) |
Subject Heading Field Tags
Database | Single Subject Heading | Multiple Subject Headings |
---|---|---|
EBSCO Most Databases EBSCO CINAHL or Medline |
DE "SubjectHeadingA" MH "SubjectHeadingA" |
DE ("SubjectHeadingA" OR "SubjectHeadingB") MH ("SubjectHeadingA" OR "SubjectHeadingB") |
PubMed | "SubjectHeading" [MH] | ("SubjectHeading" [MH] OR "SubjectHeading" [MH]) |
Web of Science | No Subject Headings | N/A |
Boolean Operators
Boolean operators are a universal type of database syntax used to group together terms and define the order of operations for a search.
AND = combine two different concepts; all results must include both concepts
OR = combine alternative terms for the same concept; all results must include at least one of those terms
Do not use NOT in systematic search design except with a validated search hedge.
Parenthesis group terms together so it is clear to the database, which 'calculations' are run in which order.
As syntax this looks like:
(PopulationTermA OR PopulationTermB OR PopulationTermC) AND (InterventionTermX OR InterventionTermY OR InterventionTermZ)
Proximity Operators
Some databases have operators that connect terms loosely by being close to each other but not necessarily in a perfect phrase structure. If "quotation marks" make a phrase highly precise, proximity operators allow for a little more natural variation in how those words are connected.
Only use these with keywords, not subject headings.
Database | Structure | Description |
---|---|---|
EBSCO Near Operator | N# | (KeywordA N4 KeywordB) = finds the keywords words if there are a maximum of 4 words between them, regardless of the order they appear in. |
EBSCO Within Operator | W# | (KeywordA W4 KeywordB) = finds the keywords words if they are a maximum of 4 words between them, but only in the order in which you enter them E.g. KeywordA must come before KeywordB. |
PubMed Proximity Search | "search terms"[field:~N] | "keyword phrase"[tiab:~5] Search terms = your keyword phrase, field = TIAB for title, abstract and keywords, and N = the maximum number of words between the keyword phrase terms (5 in my example). If you're confused (don't worry, this confuses me a lot, too), this website explains more or you can email me. |
Web of Science Near Operator | NEAR/# |
NEAR/8 find records where the terms joined by the operator are within eight words of each other. |
Additional Methods
Searching using additional methods beyond database searches enhances the comprehensiveness of the project.
In order to maintain the structure and systematic process, plan these methods in the protocol phase, identify methods that are as reproducible as possible, and transparently report all steps taken.
Where Else Do I Search?
References (backward in time)
Check all references from the papers you include in your final analysis against your selection criteria.
Cited By (forward in time)
Use Google Scholar, Web of Science or special tool like Citationchaser on all articles you selected for analysis to check for articles that cited them since they were published. Check all these articles against your selection criteria.
- CitationchaserA tool that uses LENS.org to help build an exportable "citation chain" forward and backward in time from a set of articles.
- Google ScholarTo find the articles that have cited an article since its publication, search for it by title, then click on the "cited by" link under the record for it.
- Web of Science
Coverage: Varies
Multidisciplinary database covering journal articles and conference proceedings in arts and humanities, social sciences, and sciences.
"Hand" Search Specific Journals
Select a set of journals that publish regularly on the topic of interest in your review. This may be done in consultation with research advisors, knowledgeable colleagues or consultation with a librarian. Use BrowZine, journal websites or databases to scan the titles and abstracts of all issues of that journal (either in a selected time frame or as far back as possible).
- BrowZineCoverage: Varies
Use BrowZine to easily find, read, and monitor thousands of scholarly journals available from your university or college library, or through Open Access publishers, covering all disciplines.
Dissertations and Thesis
Many valuable sources of evidence are published as dissertations or thesis. They are also often working in newer and emerging areas of study. Try running your search design through ProQuest Dissertation and Thesis (it will need to be translated to the proper database syntax). Or try DART-Europe E-theses Portal or Networked Digital Library of Dissertations & Theses (NDLT) for more international coverage.
- DART-Europe E-theses PortalA searchable database of electronic research theses held in European repositories. The theses listed are open access - publicly available, in full, without charge. The DART-Europe Portal does not store theses but aims to provide a link to at least one electronic copy of every thesis listed in the database.
- Dissertations & Theses Global (ProQuest Collection)Coverage: 1743 to present
Multi-databases search of all ProQuest Dissertations and Theses (1743-present) and including select CSU Theses & Dissertations with CSU theses (2010-present) and CSU dissertations (1950-present). - Networked Digital Library of Theses and Dissertations (NDLTD)Coverage: Varies.
The Networked Digital Library of Theses and Dissertations (NDLTD) is an international organization dedicated to promoting the adoption, creation, use, dissemination, and preservation of electronic theses and dissertations (ETDs). Topics include how to find, create, and preserve ETDs; how to set up an ETD program; legal and technical questions; and the latest news and research in the ETD community.
Clinical Trial Registries
Clinical trial registries collect the registrations of the intention and research plan for conducting a clinical trial. Not all of these end up being published, even if they produced results. It can be valuable to find records of clinical trials that were planned on your topic to see if you collected the published result or potentially contact authors to see if they are able to provide you with results for your project.
- ClinicalTrials.govThe U.S. National Library of Medicine (NLM) database of privately and publicly funded clinical studies conducted around the world.
- ISRCTN RegistryA clinical trial registry recognized by the WHO and ICMJE that accepts planned, ongoing or completed studies of any design. This includes both observational and interventional trials.
- World Health Organization ICTRP (International Clinical Trials Registry Platform) Search PortalA search portal that allows for searching across multiple international trial registries.
Google Scholar
You can just design and run a search in Google Scholar. Google Scholar does not have field tags and the search syntax is limited, so results tend to be large. It is also difficult to export results from a search. Generally, in evidence synthesis projects, you test a search in Google Scholar and review the results, decide the how many results from Google Scholar you will screen, then use a tool like Publish or Perish to run and export the search and number of results you selected.
- Publish or Perish (free)Publish or Perish uses Google Scholar and Microsoft Academic Search to analyze article and author level metrics.
Grey Literature
Grey literature is a term for publications that do go through the scholarly publication process (peer review, particularly). Large amounts of research and other data are produced and published outside scholarly channels, so try one or more relevant databases, like Mednar or OAIster.
Evidence Synthesis Librarian

Colorado State University
Fort Collins, CO 80523-1019
(970) 491-1146
anna.ferri@colostate.edu