The absence of a gold standard for measuring antibiotic use in hospitals is reflected in a diversity of indicators and recommendations (6, 7, 17). A literature review in 2015 identified 74 unique indicators, of which 12 were judged suitable by an expert panel through a consensus process (7). Among those twelve were three of the indicators evaluated in the current study (the standard indicator A1 as well as A1 p and B1). Two further indicators included in our study were considered but rejected by the expert panel (B1 p and C1).
To the best of our knowledge, our study is the first to evaluate indicators for antibiotic use in hospitals according to objective criteria. As a basis for comparison, we used a highly significant reference correlation between regional consumption rates for TZP/3GC in hospitals and the incidence of bacteraemia with ESBL-E. coli. Findings from previous studies suggest that the reference correlation reflects a causal relationship (16, 18).
We chose DDD-based consumption rate as an activity-neutral reference indicator, as it is suitable for surveillance of antibiotic use in hospitals across national borders (6). Introduction of the consumption rate as standard indicator for nationwide antibiotic use in hospitals in Norway would be consistent with European practice and facilitate cross-sectoral surveillance, including of antibiotic use in municipal health facilities (6, 8, 13). The demonstration of a strong correlation (the reference correlation) between regional consumption rates for TZP/3GC and the incidence of resistance to the same agents supports the idea that the consumption rate can also be used for surveillance and benchmarking of antibiotic use at a regional level. Ideally, the same indicator should be used at all hospital levels, but the consumption rate is not suitable for use in individual hospitals (6). This issue can be resolved through parallel use of an alternative indicator that correlates well with the consumption rate. We found that all alternative indicators met the validation criterion of a correlation between antibiotic use and resistance that was at least as strong as that observed with the standard indicator. Of the five indicators that showed a significant correlation in our evaluation, the new indicator C2 p (combined adjustment for number of patients admitted and case mix) was notable for showing the greatest accordance with the consumption rate. In contrast to the standard indicator DDD per 100 bed days, the indicator C2 p was also largely unaffected by changes in hospital activity in 2020 as a result of the pandemic.
All indicators that are constructed as fractions will be influenced by factors that affect the size of the denominator, and if such factors (for example population density and the size of the catchment area) vary between hospitals, the usefulness of the indicator for benchmarking will be reduced. This is illustrated by the fact that Northern Norway has the longest hospital stays of the regional health authorities, and the lowest use of broad-spectrum antibiotics measured with the standard indicator (3, 10). Indicators that adjust for number of bed days or inpatient admissions are also vulnerable to confounding by low efficiency and/or poor quality of care. For example, a high number of readmissions can mask high antibiotic use through the increase in size of the denominator. Conversely, improvement measures that result in fewer bed days or inpatient admissions (such as more day case admissions) will increase calculated use. The impact of such factors can be reduced by using the total number of unique admitted patients as an adjustment factor instead.
The case mix index is calculated based on hospital diagnosis coding and is part of the activity-based funding system for hospitals in Norway (11). The positive correlation between the case mix index and the use of broad-spectrum antibiotics indicates that the index reflects differences in patient complexity that affect prescribing of antibiotics. As such factors are often non-modifiable (such as the proportion of patients with infection as their primary diagnosis), use of a smoothing factor would provide a stronger basis for benchmarking (9, 19, 20). However, this principle is rarely applied in practice (7, 17). Our findings are consistent with a Swiss study that found a significant correlation between the case mix index and total antibiotic use in hospitals (9). Although the quality of DRG coding can vary between hospitals, and the case mix index does not compensate for all variation in antibiotic use related to differences in patient complexity, the findings support the use of the index as an adjustment factor.
Our findings illustrate that the current standard indicator should be replaced by other indicators that are better suited for surveillance and benchmarking of antibiotic use in hospitals. Based on the overall performance of the evaluated alternative indicators – taking into account the correlation between antibiotic use and resistance, the accordance with consumption rates, and robustness to variation in activity, quality, and efficiency between hospitals and over time – we consider the most suitable indicator to be one that combines adjustment for the total number of admitted patients and for case mix.
This study has several limitations. In the absence of prescribing data, we had to use sales figures for antibiotics. Our evaluation also relied on a single correlation between antibiotic use and resistance, and the results should be verified by follow-up studies with other reference correlations. Another limitation is that we only evaluated indicators that use DDD as a measure of total antibiotic use, despite the fact that defined daily doses do not always correspond to the doses administered (17, 19).
We conclude that the standard indicator DDD per 100 bed days underestimated the reduction in the use of broad-spectrum antibiotics in hospitals in Norway, and that the actual reduction in the period 2012–20 was close to 30 %. The novel indicator that combines adjustment for the total number of admitted patients and for case mix is reliable, robust, and suitable for hospitals at all levels. The indicator can be used in parallel with the consumption rate, and consideration should be given to introducing the latter as the new standard indicator for antibiotic use in hospitals at national and regional levels.