Glossary of GRADE-CERQual terminology


List of GRADE-CERQual terminology

Version 1, 14th August 2023

Download a copy HERE


GRADE-CERQual level of confidence

  • An assessment of the extent to which the review finding is a reasonable representation of the phenomenon of interest.(1)

High confidence in the evidence

  • It is highly likely that the review finding is a reasonable representation of the phenomenon of interest.(1)

Moderate confidence in the evidence

  • It is likely that the review finding is a reasonable representation of the phenomenon of interest.(1)

Low confidence in the evidence

  • It is possible that the review finding is a reasonable representation of the phenomenon of interest.(1)

Very low confidence in the evidence

  • It is not clear whether the review finding is a reasonable representation of the phenomenon of interest.(1)

Relevance

  • The extent to which the body of data from the primary studies supporting a review finding is applicable to the context specified in the review question. 
  • By "context" we refer to a range of factors that may interact and could include, but are not restricted to, the perspective or population, the phenomenon of interest, and the setting.
  • We are less confident that the review finding reflects the phenomenon of interest when the contexts of the primary studies underlying a review finding are substantively different from the context of the review question.
  • Concerns about relevance fall into three groups:
  1. Indirect relevance – the studies underlying the review finding do not directly reflect the review question.
  2. Partial relevance – the studies underlying the review finding only represent a subset of the review question.
  3. Unclear relevance – the extent to which the studies underlying the review finding reflect the review question is unclear.(2)*

Methodological limitations

  • The extent to which there are concerns about the design or conduct of the primary studies that contributed evidence to a review finding.
  • We are less confident that the review finding reflects the phenomenon of interest when the primary studies underlying a review finding are shown to have problems in the way they were designed or conducted.
  • The methodological limitations of contributing studies are assessed using an appropriate critical appraisal tool for qualitative studies.(3)

Coherence

  • How clear the fit is between the data from the primary studies and the review finding.
  • We are less confident that the finding reflects the phenomenon of interest when the fit between the data from the primary studies and the review finding that synthesizes these data is not completely clear. This may be because patterns in the data are not well explored or explained.(4)

Adequacy of data

  • The degree of richness and quantity of data supporting a review finding.
  • We are less confident that the finding reflects the phenomenon of interest when the data underlying a review finding are not sufficiently rich or only come from a small number of studies or participants.(5)

Dissemination bias in qualitative research

  • A systematic distortion of the phenomenon of interest due to selective dissemination of qualitative studies or the findings of qualitative studies.(6)
  • (Dissemination bias is used as a synonym to publication bias. The term dissemination bias is preferred because it widens the concept beyond publications in scientific journals.)

Review finding (in the context of a qualitative evidence synthesis)

  • A theme, category, thematic framework, theory or contribution to theory, or another similar output from a qualitative evidence synthesis, and that is based on data from primary studies.(1)*

Summary of a review finding

  • A statement or summary that provides a short but clear description of each review finding.(7)

Summary of Qualitative Findings (SoQF) Table

  • A table that includes summaries of the findings from a qualitative evidence synthesis alongside the overall GRADE-CERQual assessment for each finding. The SoQF Table also includes an explanation of the overall GRADE-CERQual assessment and references to the studies contributing to each review finding.(7)

Qualitative evidence profile

  • The qualitative evidence profile includes the same information as the SoQF Table. In addition, it includes information on the judgments the review team has made for each CERQual component underlying the overall CERQual assessment.(7)

References

(1) Lewin, S., Booth, A., Glenton, C. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implementation Sci 13, 2 (2018). https://doi.org/10.1186/s13012-017-0688-3

(2) Noyes, J., Booth, A., Lewin, S. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 6: how to assess relevance of the data. Implementation Sci 13 (Suppl 1), 4 (2018). https://doi.org/10.1186/s13012-017-0693-6 [i]

(3) Munthe-Kaas, H., Bohren, M.A., Glenton, C. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 3: how to assess methodological limitations. Implementation Sci 13 (Suppl 1), 9 (2018). https://doi.org/10.1186/s13012-017-0690-9

(4) Colvin, C.J., Garside, R., Wainwright, M. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 4: how to assess coherence. Implementation Sci 13 (Suppl 1), 13 (2018). https://doi.org/10.1186/s13012-017-0691-8

(5) Glenton, C., Carlsen, B., Lewin, S. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 5: how to assess adequacy of data. Implementation Sci 13 (Suppl 1), 14 (2018). https://doi.org/10.1186/s13012-017-0692-7

(6) Booth, A., Lewin, S., Glenton, C. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 7: understanding the potential impacts of dissemination bias. Implementation Sci 13 (Suppl 1), 12 (2018). https://doi.org/10.1186/s13012-017-0694-5

(7) Lewin, S., Bohren, M., Rashidian, A. et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table. Implementation Sci 13 (Suppl 1), 10 (2018). https://doi.org/10.1186/s13012-017-0689-

*We have made minor amendments to the definitions of these concepts since the papers were published