Subject Guides: Reviewing the literature: Appraising

Critical appraisal, or risk of bias, is done to assess if a study can be trusted. It is not required for all review types, such as scoping reviews. It is typically performed for each included study in a systematic review.

If the included studies are biased, for example if there are problems with the study design, then the results of the review may be impacted.

Covidence has built in risk of bias templates that can be customised.

There are many tools and checklists available that can be used for critical appraisal.

Covidence: Quality assessment
Critical Appraisal Skills Programme (CASP)
Checklists to help you more easily and accurately perform critical appraisal across a number of different study types.
JBI critical appraisal tools
JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers.
Centre for Evidence Based Medicine
Critical appraisal worksheets to help you appraise the reliability, importance and applicability of clinical evidence.
SIGN: Methodology checklists
These checklists were subjected to evaluation and adaptation to meet SIGN’s requirements for a balance between methodological rigour and practicality of use.
GRADE (Cochrane)
GRADE is a systematic approach to rating the certainty of evidence in systematic reviews and other evidence syntheses.
Risk of bias tools (Cochrane)
The resources in this collection mostly cover the Cochrane risk-of-bias tool for randomized trials (RoB 2).
AMSTAR
With moves to base more decisions on real world observational evidence, AMSTAR 2 should assist in the identification of high quality systematic reviews

Open Education Resource

For more information, the following PDF is the draft of an Open Education Resource currently under revision. The videos and activities are unavailable until the OER is published online.

Appraising
Roga, E., & Pruis, K. (2024). Introducing scoping and systematic reviews. Federation University Australia.

Key readings

Porritt, K., Gomersall, J., & Lockwood, C. (2014). JBI’s Systematic Reviews: Study selection and critical appraisal. American Journal of Nursing, 114(6), 47-52. https://doi.org/10.1097/01.Naj.0000450430.97383.64