From clinical questions to trustworthy answers

Per Olav Vandvik; Michael Bretthauer; Ingrid Charlotte Kongerud

doi:10.4045/tidsskr.13.0650

Perspectives

From clinical questions to trustworthy answers

Norwegian

Per Olav Vandvik, Michael Bretthauer, Ingrid Charlotte Kongerud

See All Articles

Per Olav Vandvik

Per Olav Vandvik (born 1968) is a teacher at the KLoK course at the University of Oslo. He is Associate Professor at the Department of Health Management and Health Economics, researcher at the Norwegian Knowledge Centre for the Health Services and Acting Senior Consultant at the Department of Medicine, Innlandet Hospital, Gjøvik.

The author has completed the ICMJE form and declares no conflicts of interest.

Email: per.vandvik@gmail.com

See All Articles

Michael Bretthauer

Michael Bretthauer (born 1969) is responsible for the KLoK course at the University of Oslo, Professor at the Department of Health Management and Health Economics, University of Oslo, Senior Consultant at the Section for Gastrointestinal Diseases, Department of Transplantation Medicine, Oslo University Hospital, and Research Supervisor at Sørlandet Hospital.

The author has completed the ICMJE form and declares the following conflicts of interest: He is member of the scientific advisory group for Exact Sciences and has received consultation fees from Norchip/Hjort Solicitors.

See All Articles

Ingrid Charlotte Kongerud

Ingrid Charlotte Kongerud (born 1988) is a medical student at the University of Oslo and has for two years served as research assistant at the Department of Health and Society, University of Oslo, where she has worked on a project on evidence-based practice.

The author has completed the ICMJE form and declares no conflicts of interest.

Article

Evidence-based practice is based on a combination of the best current research evidence, clinical expertise and the patients’ own preferences. In evidence-based practice, tools are used to find trustworthy answers to relevant clinical questions, and this requires skills in knowledge management. To achieve changes in clinical practice, knowledge management should be accompanied by quality improvement.

Doctors need to provide patients with a correct diagnosis and safe and effective treatment. Evidence-based clinical practice means integrating the best current research evidence with clinical, experience-based expertise and the patient’s values and preferences (1, 2). Evidence-based practice is increasingly recognised as a required part of clinical practice, but is still not very common (3) – (5). This could help explain the gap between what doctors are doing in practice and what they ought to do based on scientific documentation (6).

At the same time, clinicians are struggling to find good answers to clinical questions. In an American study, young doctors in a medical department formulated a new clinical question for every other patient (7). Answers were found to only one-third of the questions, most often through unsystematic, random sources such as colleagues and textbooks, and this concurs with the results from a study of Norwegian doctors (4). Medical textbooks have not undergone quality assurance with regard to the collection, critical appraisal and updating of the evidence base. Advice from senior colleagues is invaluable in clinical practice but should also encompass updated and research-based knowledge.

Doctors need to possess basic skills in identifying, interpreting and applying research evidence in their own practice (8). Groups responsible for developing tools for decision support, such as clinical practice guidelines, local protocols and care plans – as well as clinicians who are using these tools – are to a varying extent aware of new standards and requirements for such tools (9).

Evidence-based practice can be regarded as a toolbox that enables us to find reliable and applicable answers to clinical questions. New methods and tools make the quest for trustworthy answers easier than before. In this article we will use the question described in the introductory vignette (enteral nutrition in the case of acute pancreatitis) to describe a stepwise process – from the recognition of a need for information through the formulation of a specific question and search and critical appraisal of research evidence to integration of this evidence through clinical experience and the patient’s preferences (Figure 1). Evidence-based practice in the health services is applicable at the individual as well as the system level (10).

Figure 1: Example of a clinical question as a starting point for a stepwise process from question to answer — **Figure 1:** Example of a clinical question as a starting point for a stepwise process from question to answer

Formulation of questions

The first step is the formulation of specific questions. In formulating the question, it is necessary to recognise that we have various types of core questions in health care, and formulate these questions in a precise manner. Clinicians tend to ask questions concerning diagnosis, prognosis and treatment, but they may also have questions related to prevalence, causes of disease or the experiences, values and preferences of patients.

Different types of core questions require studies with a suitable design. The question of enteral nutrition in the case of acute pancreatitis is related to treatment, and should be answered by studies with a randomised, controlled design. A correctly performed randomisation procedure ensures that prognostically important factors that may affect the outcome are equally distributed between the patients who receive enteral nutrition and those who receive total parenteral nutrition. Other systematic errors may be caused by insufficient blinding and large attrition of patients. For questions related to the patients’ experiences, values and preferences, a qualitative study design would be more appropriate.

Precisely formulated clinical questions highlight the problem and pave the way for the literature search. A precise question should comprise information about the patient group we are studying, the interventions we would consider and the outcomes that are relevant. The questions should be formulated in the so-called PICO format, i.e. with a precise description of the patient or the population (P), treatment alternatives, i.e. interventions (I) and comparators (C), and clinical end points/outcomes (O) (11). To ensure that the treatment provides more benefit than harm, we should emphasise those end points/outcomes that are important for the patient, and keep in mind that treatments may also have side effects.

A well-formulated PICO question for the problem facing this patient could be as follows: What is the effect of enteral nutrition on mortality, duration of hospitalisation, multiorgan failure, surgical treatment, sepsis and pain when compared to total parenteral nutrition for patients with acute severe pancreatitis?

Searching for research evidence

Traditional teaching of evidence-based practice typically emphasises systematic literature searches and critical appraisal of the identified studies. As this process is time-consuming to perform and difficult to master, clinicians need to have unrestrained access to clinical practice guidelines that provide recommendations for diagnosis and treatment.

Figure 2 shows the so-called knowledge pyramid (the 6S model), which has been developed to help doctors and other health care providers quickly identify research evidence about diagnosis, prognosis and treatment (12). The model ranks information sources in a hierarchy, where the uppermost ones, such as evidence-based textbooks and guidelines, contain advice and recommendations based on pre-appraised systematic reviews, while sources further down in the hierarchy, such as systematic reviews and individual studies, require more knowledge about research methodology, including critical appraisal of the results.

Figure 2: The Knowledge Pyramid, a hierarchy of information resources for clinical questions — **Figure 2:** The Knowledge Pyramid, a hierarchy of information resources for clinical questions

Searching the literature has been simplified through web-based services such as The Health Library and Pyramid Searches. The Health Library (www.helsebiblioteket.no) is a web portal that includes more than 2 500 journals, medical databases, clinical guidelines and reference manuals which are freely available to health personnel (13). The front page displays the Pyramid Search, which quickly provides answers at various levels of the knowledge pyramid with the aid of one or two search terms from the PICO question. Moreover, through the search engine McMaster PLUS, the pyramid search will help you find pre-appraised systematic reviews and individual studies that are deemed to be valid and interesting for clinical practice. The selected studies are easily available through e-mail alerts, which can be received after registering with the Health Library. Such e-mail alerts provide a good opportunity to keep updated in the field without being inundated by a flood of new studies.

e-Figure 3 shows the results from a pyramid search for the question concerning enteral nutrition in the case of acute pancreatitis. The surgeon entered the search term «acute pancreatitis» in the search field, and within two minutes she had found a relevant recommendation in the evidence-based textbook UpToDate. Here, a GRADE 1B recommendation for use of a nasojejunal tube was given, i.e. a strong recommendation (indicated by the figure 1) with a moderate quality of the evidence (indicated by the letter B).

Figure 3: Pyramid search for the question on acute pancreatitis, with a recommendation from UpToDate (via www… — **Figure 3:** Pyramid search for the question on acute pancreatitis, with a recommendation from UpToDate (via www.helsebiblioteket.no)

The surgeon is surprised. Even though she is somewhat uncertain of what is meant by GRADE 1B, her interpretation is that the patient should be provided with a nasojejunal tube, not with total parenteral nutrition. This contradicts prevailing practice in the department. She therefore wants to know more about the basis of this recommendation, and with one click in UpToDate she finds a text section that refers to a systematic review with documented positive effects of enteral nutrition on the outcomes that are important to the patient (14).

Critical appraisal

Doctors should be able to critically assess guidelines and research results before clinical practice is changed. The purpose of a critical appraisal is to make sure whether one can trust, understand and apply guidelines or studies in practice (15). The online training course www.kunnskapsbasertpraksis.no provides a more detailed introduction to critical appraisal of professional guidelines, systematic reviews and primary studies with varying types of design. The course includes checklists as tools to ask the right questions about matters such as the reliability of the studies.

The GRADE system (Grading of Recommendations Assessment, Development and Evaluation) (e-Box 1) is an alternative to traditional critical appraisal with the aid of checklists (16). This is a method for the systematic and transparent appraisal of the quality of research-based knowledge and for the development of professional guidelines. The GRADE system has been developed by organisations that participate in the development of guidelines and systematic reviews, including the Cochrane Collaboration. The Norwegian Directorate of Health recommends the use of GRADE for development of professional guidelines (17).

BOX 1

GRADE (Grading of Recommendations Assessment, Development and Evaluation)

System for quality assessment of research evidence and development of professional guidelines (www.gradeworkinggroup.org/society/)

GRADE defines quality of evidence as confidence in the results (effect estimates) generated by the studies (16). The five following factors¹ may reduce the quality of the documentation from high to low:

Risk of bias: Was the randomisation concealed, who was blinded, was there major attrition?
Consistency: Are the results consistent across individual studies in the meta-analysis?
Applicability: Can we transfer the results to our clinical practice?
Precision: How precise are the effect estimates as expressed by the confidence intervals?
Publication bias: May key studies have remained unpublished?

In the development of recommendations in clinical practice guidelines, GRADE emphasises a systematic process based on an integrated assessment of four factors when moving from evidence to either strong or weak recommendations:

The balance between desirable and undesirable consequences of the intervention studied, expressed in absolute effect estimates.
Quality of the evidence: How much confidence do we place in the effect estimates?
The patient’s values and preferences: What would patients have chosen if they had been well informed about desirable and undesirable consequences of the treatment alternatives?
Costs: Does this treatment represent a sensible use of healthcare resources?

The strength of the recommendation reflects the following:

A strong recommendation indicates that the desirable consequences clearly outweigh the undesirable consequences, and is based on the assumption that virtually all well-informed patients most likely would have opted for this treatment.
A weak recommendation indicates a finer balance between advantages and disadvantages, and is based on the assumption that most well-informed patients would have opted for this treatment.

¹ Applies to studies with a randomised, controlled design. Observation studies start at low quality, but may end up with high quality on the basis of three factors that increase our confidence in the effect estimates.

Evidence-based textbooks and guidelines are intended to provide balanced recommendations by integrating research evidence and clinical expertise, patient preferences and other contextual factors, for example availability of resources (17). New standards and criteria for guidelines require, for example, a systematic review of the evidence and a balanced assessment of advantages and disadvantages of various treatment alternatives (18, 19). The development of guidelines is a demanding task. This can be illustrated by a study of a random sample of international guidelines, less than half of which complied with established criteria for trustworthiness (9). The situation is no better with regard to Norwegian guidelines and local protocols.

Well-executed systematic reviews that summarise existing research evidence through meta-analyses provide the best knowledge base for the recommendations found in guidelines and should be used as a basis for clinical decisions in the absence of trustworthy guidelines. Criteria for critical appraisal of guidelines include clearly formulated questions, explicit inclusion criteria, systematic literature searches, critical appraisal of individual studies and summaries of results with the aid of adequate methods, such as meta-analyses. Effect estimates from meta-analyses are shown as a square diamond and provide the relative effect estimate for treatment (e-Figure 4).

Figure 4: Meta-analysis of enteral nutrition (EN) versus total parenteral nutrition (TPN) in the case of acute pancreatitis… — **Figure 4:** Meta-analysis of enteral nutrition (EN) versus total parenteral nutrition (TPN) in the case of acute pancreatitis. From Al-Omran and collaborators (14)

Individual studies are placed at the bottom of the knowledge pyramid, since they are not collated with other studies and since they also may include sources of error or other factors that limit their applicability in clinical practice (20). Nevertheless, individual studies may occasionally – after a systematic review of the documentation and a critical appraisal – constitute the best knowledge base for clinical practice (21).

The strong recommendation made by UpToDate for providing enteral nutrition through a nasojejunal tube to our patient Eva who suffers from pancreatitis is based on a well-executed, systematic Cochrane review and meta-analysis. The status of the Cochrane review as the best current research evidence for the question on enteral nutrition in the case of pancreatitis is confirmed by its position high up in the knowledge pyramid, at the level below evidence-based textbooks and guidelines (e-Figure 3).

e-Figure 4 shows a forest plot from the Cochrane meta-analysis for the outcome of mortality. Enteral nutrition in the case of pancreatitis results in a relative risk reduction of 50 % and an absolute risk reduction of 8 % (number needed to treat, NNT = 12) when compared to total parenteral nutrition. The surgeon interprets this as an impressive effect of enteral nutrition, in terms of relative as well as absolute effect.

Figure 5: GRADE Summary of findings table for the question on enteral nutrition in the case of acute pancreatitis. From Al… — **Figure 5:** GRADE Summary of findings table for the question on enteral nutrition in the case of acute pancreatitis. From Al-Omran and collaborators (14)

The absolute effects of a treatment are often less impressive than the relative ones. A prime example is provided by mammography screening, whose introduction reduced the relative risk of death from breast cancer by 10 %. The corresponding relative risk reduction, however, amounts to less than 0.5 % (22). As important as being able to interpret relative and absolute effects is the ability to balance advantages and disadvantages by identifying effects on other outcomes that are important for the patient.

The surgeon decides to read the Cochrane review, and quickly retrieves the results of the meta-analysis summarised in a «GRADE Summary of Findings table». e-Figure 5 shows an excerpt from this table, in which the authors use the GRADE system to summarise relative and absolute effects of treatment and the quality of the evidence across the outcomes that are important for the patient.

Application and evaluation of new knowledge in practice

The final step in evidence-based practice is to apply new knowledge and evaluate this change of practice. This requires implementation of systematic efforts for quality improvement (23).

The surgeon who discovered that enteral nutrition was superior to total parenteral nutrition in the case of acute pancreatitis finds support in a strong recommendation from UpToDate and claims that practice ought to be changed, in light of the dramatic gains provided. Her colleagues agree, but remain uncertain about how to proceed.

Systematic quality improvement is only to a minor extent integrated into clinical work and specialist training of doctors. However, good training methods, tools and systems are available (23). Medical students at the University of Oslo learn about evidence-based practice and systematic quality improvement through the course «Knowledge management, leadership and quality improvement», referred to as the KLoK course (24). The Ministry of Health and Care Services and the Norwegian Directorate of Health have proposed to introduce mandatory training in evidence-based practice and also quality improvement in specialist training (23).

A group of medical students in Oslo have shown how a local hospital can implement enteral nutrition for patients with acute pancreatitis (25). The students undertook this assignment during their studies and in contact with the department of surgery, which now has good opportunities to implement this intervention in practice.

Training in knowledge management

Substantial training is required to develop sufficient skills in knowledge management (8). The medical curriculum at the University of Oslo now includes such training (24), for example through clinically integrated learning with the aid of work files (25). In an article in this issue of the Journal of the Norwegian Medical Association we describe how medical students in Oslo learn knowledge management through clinically integrated learning with the aid of such work files (26). Teaching of knowledge management is also on its way into the medical curriculum in Tromsø and Trondheim, and has been proposed as a key teaching module in post-graduate studies in Norway. Furthermore, training courses in knowledge management for hospital doctors have been developed, as well as a Nordic workshop for all professional groups in health care. In the meantime, the online training course www.kunnskapsbasertpraksis.no can be used to acquire practical skills in knowledge management. In this way, we can ensure that doctors do more good than harm through adequate diagnostics and safe and effective treatment.

Literature

1.
Sackett DL, Rosenberg WM, Gray JA et al. Evidence based medicine: what it is and what it isn’t. BMJ 1996; 312: 71 – 2. [PubMed] [CrossRef]
2.
Wyller TB. Hvilken kunnskap om kunnskapen? Tidsskr Nor Legeforen 2011; 131: 1741. [PubMed]
3.
Guyatt G, Cook D, Haynes B. Evidence based medicine has come a long way. BMJ 2004; 329: 990 – 1. [PubMed] [CrossRef]
4.
Ulvenes LV, Aasland O, Nylenna M et al. Norwegian physicians’ knowledge of and opinions about evidence-based medicine: cross-sectional study. PLoS ONE 2009; 4: e7828. [PubMed] [CrossRef]
5.
Vandvik PO, Eiring O. Kunnskapsbasert praksis for norske leger? Tidsskr Nor Legeforen 2011; 131: 1755. [PubMed]
6.
Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet 2003; 362: 1225 – 30. [PubMed] [CrossRef]
7.
Green ML, Ciampi MA, Ellis PJ. Residents’ medical information needs in clinic: are they being met? Am J Med 2000; 109: 218 – 23. [PubMed] [CrossRef]
8.
Glasziou P, Burls A, Gilbert R. Evidence based medicine and the medical curriculum. BMJ 2008; 337: a1253. [PubMed] [CrossRef]
9.
Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet institute of medicine standards: Two more decades of little, if any, progress. Arch Intern Med 2012; 172: 1628 – 33. [PubMed] [CrossRef]
10.
Vandvik PO, Eiring O. Foretaksprosjektet: mot kunnskapsbasert praksis i spesialisthelsetjenesten. Oslo: Nasjonalt kunnskapssenter for helsetjenesten, 2011.
11.
Guyatt GH, Oxman AD, Kunz R et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol 2011; 64: 395 – 400. [PubMed] [CrossRef]
12.
DiCenso A, Bayley L, Haynes RB. Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model. Evid Based Nurs 2009; 12: 99 – 101. [PubMed] [CrossRef]
13.
Nylenna M, Eiring O, Strand G et al. Wiring a nation: putting knowledge into action. Lancet 2010; 375: 1048 – 51. [PubMed] [CrossRef]
14.
Al-Omran M, Albalawi ZH, Tashkandi MF et al. Enteral versus parenteral nutrition for acute pancreatitis. Cochrane Database Syst Rev 2010; 1: CD002837. [PubMed]
15.
Guyatt GH, Haynes RB, Jaeschke RZ et al. Users’ Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users’ Guides to patient care. JAMA 2000; 284: 1290 – 6. [PubMed] [CrossRef]
16.
Guyatt GH, Oxman AD, Kunz R et al. What is «quality of evidence» and why is it important to clinicians? BMJ 2008; 336: 995 – 8. [PubMed] [CrossRef]
17.
Veileder for utvikling av kunnskapsbaserte retningslinjer. Oslo: Helsedirektoratet, 2012.
18.
Laine C, Taichman DB, Mulrow C. Trustworthy clinical guidelines. Ann Intern Med 2011; 154: 774 – 5. [PubMed] [CrossRef]
19.
Qaseem A, Forland F, Macbeth F et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med 2012; 156: 525 – 31. [PubMed] [CrossRef]
20.
Ioannidis JP. Why most published research findings are false. PLoS Med 2005; 2: e124. [PubMed] [CrossRef]
21.
Vandvik PO, Lincoff AM, Gore JM et al. Primary and secondary prevention of cardiovascular disease: antithrombotic therapy and prevention of thrombosis. 9. utg. American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012; 141 (suppl): e637S-68S. [PubMed] [CrossRef]
22.
Hem E. Effekter av og ulemper ved mammografiscreening. Tidsskr Nor Legeforen 2013; 133: 508.
23.
Meld.St. nr.10 (2012 – 2013). God kvalitet – trygge tjenester. Kvalitet og pasientsikkerhet i helse- og omsorgstjenesten.
24.
Frich JC, Gran SF, Vandvik PO et al. Kunnskap, ledelse og kvalitet i studiet. Tidsskr Nor Legeforen 2012; 132: 1768 – 70. [PubMed]
25.
Haslestad SF. TM, Wibetoe G, Høyset B et al. Implementering av evidensbasert ernæringsregime ved akutt alvorlig pankreatitt. Prosjektoppgave. Oslo: Institutt for helse og samfunn, Universitetet i Oslo, 2012.
26.
Kongerud IC, Vandvik PO. Arbeidsfiler som læringsverktøy i kunnskapshåndtering. Tidsskr Nor Legeforen 2013; 133: 1587 – 90.

Comments ( 0 )

Dette kommentarfeltet modereres, men kommentarer blir ikke redaksjonelt behandlet ut over å sikre at de følger retningslinjer for vårt kommentarfelt.

This article was published more than 12 months ago and we have therefore closed it for new comments.

Published: 20 August 2013

Tidsskr Nor Legeforen 20 August 2013

doi:

10.4045/tidsskr.13.0650

Received 13 May 2013, first revision submitted 5 June 2013, approved 20 June 2013. Medical editor: Petter Gjersvik.

133

:

1611-4

Published: 20 August 2013

Tidsskr Nor Legeforen 2013

133

:

1611-4

doi: 10.4045/tidsskr.13.0650

Received 13 May 2013, first revision submitted 5 June 2013, approved 20 June 2013. Medical editor: Petter Gjersvik.

PDF

Print

From clinical questions to trustworthy answers

Formulation of questions

Searching for research evidence

Critical appraisal

GRADE (Grading of Recommendations Assessment, Development and Evaluation)

Application and evaluation of new knowledge in practice

Training in knowledge management

Recent Articles