Systematic Reviews & Evidence Synthesis

Why You Need a Research Team

A minimum of three (3) individual members is recommended to fill the following roles:

2 Screeners

Two (2) individuals screen every reference, blinded from each other's decisions

1 Tie-Breaker

One (1) individual acts as a tie-breaker when resolving disagreements between the two screeners

This same structure should generally be used for data extraction and for quality assessment. An alternative to having a designated tie-breaker is to plan (during the protocol phase) a process for reaching consensus between the two screeners that mitigates bias.

For larger projects, it can be useful to have even more screeners. Ideally the same screeners review every result, but it is possible to break the results into smaller sets for a larger group of screeners as long as:

1. every article is separately screened by two people, and
2. there is a process for all screeners to be trained and calibrated on the application of inclusion/exclusion criteria

The purpose of this process is to reduce bias and errors, thereby increasing the methodological rigor. A systematic review management or screening application can help!

Team Member Roles

Multiple expert roles may be filled by a single individual with corresponding skills. Other team members may work on screening, data extraction, and/or quality assessment, but not fill any of these expert roles.

• Subject Expert: Strong familiarity and comfort with the field(s) and literature related to your research question(s)
• Methodological Expert: Strong familiarity and comfort with the evidence synthesis methodology, ideally including prior evidence synthesis research team experience
• Project Manager: Ensures the project stays on track (timeline, budget, and methodological expectations); ideally with experience in methodological requirements, software, and tools for evidence synthesis
• Information Retrieval Expert: Expertise in designing, executing, and documenting a comprehensive search strategy; familiarity and comfort with platforms and structures for relevant databases and systematic practices for non-database searches
• Statistician: Familiarity with statistical synthesis methods including meta-analyses; ideally having experience running meta-analyses and/or as a member of a different systematic review and meta-analysis team
• Community stakeholders: Patients, consumer advocates, practitioners, community groups, public officials, professional organizations and other stakeholders should be engaged throughout the review, especially during the planning phase

Communications & Documentation Plan

As you gather the research team, take time to identify:

• How you will your team communicate and what timelines are expected for responses?
• Who is involved at each phase, what are their responsibilities, and how will you manage unexpected changes to work loads?
• How will documents be shared and who will have access to which components?
• How will versions of documents be labeled, dated, and organized so you can track and report on decisions, process and progress?
• What software will be needed at each phase, who needs access to it, and what funds do you have for software needs?
• Who on the team is responsible for ensuring you collect all necessary methodological information to meet reporting guidelines for the final product?

Evidence synthesis projects take a lot of time to plan and to conduct. It can be hard to estimate a project timeline. It is best to have a flexible timeline for many evidence synthesis projects.

What Factors Influence Your Timeline

1. Topic of the evidence synthesis project (scope and complexity)
2. Number of team members, their availability, and their experience levels
3. Variety of new tools, techniques, and resources you need access to and training on
4. Type & methods of evidence synthesis project selected
5. Availability of full-text and amount of contacting authors required
6. Need to get articles in other languages translated
7. Access to support

Extra time and care during your planning and protocol writing phases can make later parts of the process run much more smoothly, efficiently, accurately and with less risk of bias.

Suggested Timelines

These are rough estimates based on reviews from various fields on the time it takes to complete an evidence synthesis project.