Altogether 90 of the 91 work files described understandable and relevant clinical problems. A total of 76 work files described effects of treatment, while the remaining focused on diagnostics and prognosis.
On the basis of the clinical problems, the students developed focused questions in PICO format: 70 work files contained all four components; in 14 of them one element was missing and in seven of them two elements were missing. Population, intervention and control group were satisfactorily defined in 84, 85 and 72 work files respectively. As regards definition of patient-related outcomes, 13 work files failed to define an outcome, whereas 60 files defined one outcome, 17 files defined two and one file defined 4 – 5 outcomes. Figure 1 illustrates that the student identified only one relevant outcome.
Before the students undertook the literature search, they were asked to identify the sources that they would normally use to find answers in their usual practice. Altogether 31 would have asked experienced colleagues, 24 would have looked in textbooks, 19 would have searched the Internet and 15 would have searched for local guidelines.
In the 91 work files, literature searches were most frequently undertaken as a pyramid search in McMaster PLUS (n = 42), followed by direct searches in Up To Date or Best Practice (n = 17), PubMed or other databases (n = 15) and searches in unnamed databases (n = 17). Answers were most frequently found in evidence-based textbooks or guidelines (n = 43), followed by primary studies (n = 30) and systematic review articles (n = 15). Only three students found answers in unreported sources.
Altogether 78 of the 91 students performed a critical appraisal of the results. In the assessment of relevant issues related to validity, 29 files contained no issues, 12 contained one issue, eight contained two issues and 29 contained three or more issues. At least one effect estimate was reported in 46 work files, of which ten files included absolute as well as relative effect estimates. As regards the assessment of whether the intervention was applicable in their own practice, altogether 58 work files reported at least one relevant issue.
A total of 89 of the 91 students claimed to have found an answer to their question. We concluded that 84 of these provided an answer with an adequate and satisfactory conclusion. Figure 1 shows how this student misinterpreted the documentation and how the professor communicated this back to her.
50 of the 91 students reported what was common treatment practice in the department where they spent their work placement period. In 17 work files, the student concluded that existing practice should be changed. This included some students who failed to describe the common treatment practice in their department
None of the students reported to have contacted a professor or a librarian during their work with the file. Only five of them reported how many hours they had spent on working with the file. Figure 1 illustrates how this student received feedback on elements in the work file that needed improvement.