Researchers in Birmingham have devised a new and stronger method of calculating and comparing hospitals’ mortality rates.

The findings were published by the British Medical Journal Open days before the Francis Report was published into the large number of unexpected deaths at Mid Staffordshire NHS Trust.

The research paper, which includes among its authors University Hospitals Birmingham NHS Foundation Trust Director of Informatics Daniel Ray and Cardiac Surgeon and Clinical Director of the Quality and Outcomes Research Unit Domenico Pagano, aims to find a more accurate measure of hospital mortality ratios.

Entitled “Can we update the Summary Hospital Mortality Index (SHMI) to make a useful measure of the quality of hospital care?”, the SHMI being the Department of Health latest national death statistic, the report draws on more than 23 million admissions to NHS Trusts in England.

Using a new method called Quality and Outcomes Research Unit Measure (QUORUM) instead, it concluded that no hospital trust had a mortality that was greater than three standard deviations above the mean figure, meaning no hospitals were outliers in this new strong death statistic.

However, the report said you might reasonably expect to see some variation in mortality in an organisation such as the NHS and, if this is the case, it questions whether using this methodology is fit for purpose to assess hospital quality of care. Assessing hospital performance using a single surrogate of hospital mortality has several limitations.

A surrogate measure may be a composite of areas where an institution has poor performance and areas where performance is excellent. The surrogate computation could show the institution performing within accepted limits, while masking clinical areas of poor performance. It may be more appropriate to focus on indicators of performance in important areas such as those promoted in the NHS Quality and Outcomes Framework.

Furthermore, using overall hospital statistics does not provide information on the quality of the individual services, a measure possibly of more interest to patients, healthcare commissioners, and regulators.

Researchers set out to develop a more efficient model with additional explanatory variables using more sophisticated statistical techniques to assess trust performance.

Its authors concluded: “We have developed a new measure (QUORUM), which directly estimates the performance of hospital trusts using all-cause mortality (in hospital or within 30 days of hospital discharge), from individual patient data, as an alternative to the recently proposed SHMI.

“Another potential strength of our work is that we took into account post-discharge outcomes and readmissions so that in many circumstances one patient death could not be counted multiple times, as with the old HSMR (Hospital Standardised Mortality Ratio) method.”

In the authors’ view, future developments should concentrate upon driving healthcare providers to collect more clinically relevant data centrally, for example early warning scores, and medicine prescribing information.

They said: “The NHS should stop focusing only on death rates as a key marker of quality. Death is important but we also need to focus on the 96.5 per cent of patients that leave hospitals alive.

“The NHS needs to collect more clinically relevant data about its patients – for example, are a full set of observations taken every 24 hours, and are patients getting the right drugs at the right time? These are basic aspects of care that should be delivered by hospitals but are not submitted electronically covering all patients to the central NHS or have targets set against them. These may contribute significantly to hospital mortality, and have a significant impact on the long term outcome of patients.”

In summary, current mortality indicators are inconsistent and flawed, the report concluded. Having developed a strong statistical model that has a proven accuracy level and looks at the death rates across the Trusts in the UK, the authors have found that no hospital has a higher than expected mortality rate.

“This research is not saying that Trusts shouldn’t deliver care better or that death is not important, but should not use death rates as the only definition of good and bad care, as it may only pick up extreme cases,” the authors said.