Content analysis of guidelines

Content analysis of guidelines

Jako Burgers, Julia Bailey, Niek Klazinga, Akke van der Bij, Richard Grol, Gene Feder, for the AGREE Collaboration.

Background

Over the past twenty years guidelines have been developed to bridge the gap between research and practice (1). There has been a concerted effort to base clinical decisions on research evidence (2) and to make this evidence available globally (3). Since bibliographic databases (i.e. Medline) are easily available, one might expect that this would lead to international consensus on the evidence chosen to underpin recommendations for clinical care, and a consequent convergence of recommendations made in guidelines. Nevertheless, recommendations often differ in guidelines on the same topic. Investigators hypothesise that differences are due to insufficient evidence (4,5), differing interpretations of evidence (6), unsystematic guideline development methods (7,8), the influence of professional bodies (9), cultural factors such as differing expectations of apparent risks and benefits (4,10), socio-economic factors or characteristics of health care systems (11).

In this study we compared recommendations between a range of guidelines on the management of type 2 diabetes, an example of a common condition with evidence of variation in practice despite a substantial body of treatment trials and other studies. We aimed to analyse to what extent the variation (or concordance) between recommendations was explained by the evidence cited in the guidelines.

Methods

We analysed 15 clinical guidelines on type 2 diabetes from 12 countries involved in the AGREE Collaboration and Australia. Qualitative methods were used to compare the recommendations and bibliometric methods to measure the extent of overlap in citations used by different guidelines. A further qualitative analysis of recommendations and cited evidence for two specific issues in diabetes care (use of metformin in obese patients and self-monitoring of blood glucose) explored the apparent discrepancy between recommendations and evidence.

Results

The recommendations made in the guidelines were in agreement about the general management of type 2 diabetes, with some important differences in treatment details. There was little overlap in evidence cited by the guidelines, with 18% (185/1,033) of citations shared with any other guideline, and only 10 studies (1%) appearing in six or more guidelines. The measurable overlap in evidence between guidelines increases if multiple publications from the same study and the use of reviews are taken into account. Research originating from the U.S. predominated (40% of citations); however, nearly all (11/12) guidelines were significantly more likely to cite evidence originating from their own countries.

Conclusions

Despite the variation in cited evidence and preferential citation of evidence from a guideline's country of origin, we found a high degree of international consensus in recommendations made for the clinical care of type 2 diabetes. The influence of professional bodies such as the American Diabetes Association may be an important factor in explaining international consensus. Globalisation of recommended management of diabetes is not a simple consequence of the globalisation of research evidence.

Note

The paper presenting the complete results of this study has been published as "Inside guidelines: comparative analysis of recommendations and evidence in diabetes guidelines from 13 countries" in Diabetes Care 2002; 25(11):1933-1939.

References

1) Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ 1998; 317:465-468.

2) Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence-based medicine: what it is and what it isn't. BMJ 1996; 312:71-72

3) Cochrane Collaboration: The Cochrane Library [database on disk and CD-ROM]. Oxford, Update Software 2002, Updated quarterly.

4) Fahey TP, Peters TJ: What constitutes controlled hypertension? Patient based comparison of hypertension guidelines. BMJ 1996; 313:93-96.

5) Vogel N, Burnand B, Vial Y, Ruiz J, Paccaud F, Hohlfeld P: Screening for gestational diabetes: variation in guidelines. Eur J Obstet Gynecol Reprod Biol 2000; 91:29-36.

6) Ravago TS, Mosniam J, Alem F: Evaluation of Community Acquired Pneumonia Guidelines J Med Sys 2000; 24:289-296.

7) Thomson R, McElroy H, Sudlow M: Guidelines on anticoagulant treatment in atrial fibrillation in Great Britain: variation in content and implications for treatment BMJ 1998; 316:509-513.

8) Beck C, Cody M, Souder E, Zhang M, Small GW: Dementia Diagnostic Guidelines: Methodologies, Results and Implementation Costs. J Am Geriatr Soc 2000; 48:1195-1203.

9) Littlejohns P, Cluzeau F, Bale R, Grimshaw J, Feder G, Moran S: The quality and quantity of clinical practice guidelines for the management of depression in primary care in the UK. Br J Gen Pract 1999; 49:205-210.

10) Eisinger F, Geller G, Burke W, and Holtzman NA: Cultural basis of differences between US and French clinical recommendations for women at increased risk of breast and ovarian cancer. Lancet 1999; 353:919-920.

11) DeMaeseneer J, Derese A: European general practice guidelines: a step too far? Eur J Gen Pract 1999; 5:86-87.