PBHL575: Epidemiological Research for Public Health

After deciding that information is relevant to your research, you need to decide whether or not it is good and reliable.

The hierarchy of evidence, or levels of evidence, helps summarize which types of studies have the strongest clinical applicability and credibility. Systematic reviews and meta-analyses are considered the strongest form of evidence, as they are essentially studies of previously done studies. Experimental studies follow with randomized controlled trials (RCTs) and then observational (non-experimental) studies.

Please note that the credibility of the review or study still depends on the methodology the reviewers/researchers took to produce it and that studies must be appraised individually. When deciding whether or not a systematic review or RCT is reliable, you still need to look at the design and methodology for that particular review or study to decide whether or not it is truly reliable.

An alternative evidence hierarchy: Pyramid of Evidence (Peggy Schmidt, VCNA: Small Animal Practice. 2007. 37(3), pp. 409-417). 

Research study methodologies

Questions to ask when assessing research methodology:

• Is the research question and hypothesis clearly defined?
• Are the methods and measurements detailed enough so that the study is reproducible?
• Were the comparison groups truly comparable?
• Were patients in each group randomized?
• Was the study blinded or double-blinded? 
• What was the dropout rate and was the dropout rate explained?
• Was the timeline long enough to allow for all possible or relevant outcomes?
• Were the comparison groups treated equally? 
• Did the authors disclose funding sources?

There are various acronyms, organizations, and guidelines that are widely used to assess research study methodologies and quality. The EQUATOR Network provides detailed information and links for each study type. Below is an abbreviated list. 

Below is also a link to checklists provided by the Joanna Briggs Institute to help evaluate different study types. 

Assessing health and medical information on the web can be tricky at times and there are no hard-and-fast rules about what is 100% credible, what may be credible (but requires further investigation), and what is not credible.

Below are some guidelines that can be used to gauge a site's credibility, but all of them exist within the context of your expertise and ability to think critically about the information presented (and how it is presented).

Health on the Net (HON) HONcode Principles

Health on the Net (HON) offers certification for medical and health websites that meet certain criteria. While it is unlikely that veterinary websites participate in HON certification, the HONcode Principles still serve a good guidelines when appraising information online.

1. Authority
   • Who created this information? Did they disclose their qualifications and credentials? Are their qualifications and credentials from a credible source?
2. Complementarity
   • Does the site state that it "is designed to support, not replace, the relationship that exists between a patient and the patient's physician?" Does the site seek to complement physician advice or to replace it? 
3. Privacy policy
   • Does the website comply with maintaining the privacy and confidentiality of patients and site visitors?
4. Attribution and date
   • Does the website or article disclose or link out to source material and data?
   • Is the website current? (Hint: look at the copyright or last modified notice at the bottom of the page or the dates for the most recent postings)
5. ​Justifiability
   • Are arguments balanced, presenting evidence for both sides?
   • Does the material seem to focus on one bias and ignore all others (or immediately dismiss) without explanation?
   • Are references and sources provided with attributions or information on where to find them? (See Principle #4)
6. Transparency
   • Do the site creators or authors provide information on how to contact them or find further support?
7. Financial disclosure
   • Does the site or its authors disclose funding sources or sponsorships (both commercial and not for profit entities)?
8. Advertising policy
   • Are ads used as a source of funding? 
   • Are ads easy to differentiate from the site source material? 
   • What kinds of products or services are featured in the ads?
   • Do the ads display inherent bias? 

Source: Health on the Net Since 1995. Certification: The HONcode Principles. 2018. Accessed from https://www.hon.ch/en/certification.html