Systematic Reviews & Evidence Synthesis

A protocol is:

• a written plan for each step of the project process, including:
  • how each step will be done
  • the reasoning for why that specific process will be used
    • specifically in relation to the goals/aim of the project and the project resources
• completed before the formal start of the project (generally, this is identified as whenever the search is run that produces the specific set of results you will screen)
• often registered publicly in a version controlled manner, in order to:
  • increase the transparency of the methods/process used
    • allow for easier replicability and/or updating of the project and
    • interpretation by audiences of the applicability of the project aims, research question and process to their needs

It's also a love letter to your future self!

These project take a lot of planning, decision making and time to complete. You will forget what you did or decided early on unless it is clearly written out. If you do most of the decision making during the planning process and record that in a thorough and clear protocol, you can move more easily through the project steps and deal more quickly with unexpected challenges.

1. Review protocol writing guidelines
2. Develop or select a protocol template
3. Draft the protocol
4. Have your protocol peer reviewed [formally or informally]
5. Finalize the protocol
6. Register or publish the protocol publicly

Guidelines

These guidelines help identify what you want to record in a protocol. Use them with discretion - your project is unique based on the research aims, question and audience for which you are designing the project..

PRISMA-Protocol guidelines are for systematic reviews and meta-analyses, but JBI has guidance on adapting these guidelines for scoping reviews. You will need to adapt these guidelines for other review types. Most methods guidelines also have protocol guidance.

Templates

Templates should work alongside the guidelines to help you structure and complete the protocol.

Most templates are designed to be used by all types of evidence synthesis projects, which means some sections will not be relevant or need to be adapted for the specific project type you are conducting.

Public preregistration of a project protocol stems from the movement towards transparency, bias reduction, and accountability in open science.

Public preregistration of a protocol is required for publication of a systematic review for many journals. It is advised as a best practice for all evidence synthesis projects and some publishers will require it for a wide variety of review types.

Where to Register a Protocol

Identify where you would most like to publish both the final manuscript and the protocol as early in the project as possible.

Some publishers require preregistration in specific places and formats. Some protocol registries have template and guidelines for you to follow.

For most projects, registering in OSF Registries (see below) is sufficient unless the publication venue of your choice suggests otherwise.

Discipline/Field Specific Registries

General Registries

Publication as Registration

There are some journals that will publish a protocol as a stand-alone article. These journals are generally expecting to also publish the final manuscript on completion of the project, and this process is known as 'registered reports.' A benefit of this process is that the protocol will go through peer review, providing additional opportunities for expert review and feedback.

Student Review Project Registration

Registration of a protocol is still recommended for thesis/capstone/dissertation projects that are not seeking further publication, but is ultimately at the discretion of program requirements or the faculty research advisor.

In these cases, the easiest venue for registration is the CSU digital repository, Mountain Scholar.

Registration or Preregistration?

Registration = Publishing the protocol publicly at any point after the start of the project / date analysis, generally done for publication purposes

Preregistration = Publishing the protocol, publicly and/or in a version-controlled manner, before the start of the project or at least before the start of data analysis

Check with you intended publication venue. They may require preregistration or just registration or neither. If you do not know where you are planning to publish, preregister to be more thorough.

Can I be scooped or denied publication if I register a protocol?

Generally this is unlikely. Doing these projects properly takes a ton of time and resources. If someone publishes a rushed project on the same topic ahead of you, then it is entirely possible to get a paper published that points to and addresses the faults of previously published, poor quality reviews.

However, if this is a concern, some registration sites (such as OSF) allow you to embargo the registration. This means the entire protocol is version controlled but only the title and authors are visible to the public. Check with the registration venue.