The current proposal by NHSE to replace the 4hr A&E target is muddled and confusing. It isn't strongly supported by evidence. It will, most likely, make the experience of patients in A&E even worse. Worse still, if things do get worse we won't have the available public data to measure the deterioration. If they paid any attention to history or the experts in emergency medicine, they could have done a much better job.
What is in the proposals?
As part of a review of NHS Access Standards NHSE propose replacing the current A&E target (95% of all patients to leave A&E inside 4hr) with a set of new targets. Those new targets are:
- Time to initial assessment
- Time to treatment for conditions requiring urgent treatment
- Mean time spent by all patients in A&E
- Utilisation of same-day emergency care
It is a little unclear what the last one of these means, but it looks like it will be based on some metric like the proportion of admissions with a zero length of stay (which the report seems to want to encourage as a way of avoiding overnight stays from admission).
What is the rationale for this?
The claimed rationale is to "measure what's most important clinically, and to patients". In support of this, the review suggests the current standard fails on several important grounds:
- It doesn't measure total waiting time
- It doesn't differentiate between severity of condition
- It measures a single point on a complex pathway
- Hospitals' processes, rather than clinical judgement are rushing discharge or admission decisions
- The standard is not well understood by the public
The idea being, at least partially, that the new standards will "drive better outcomes", drive improvement and reduce the opportunity for gaming of current targets.
All that sounds reasonable so what is the problem?
There are multiple problems with the new proposals and they fall into several categories:
- The development ignored the most relevant expert advice
- Previous work (which did use input from experts who knew their stuff) has been ignored
- The evidence used to support the new proposals is weak
- Some of the thinking in the report seems to be naive to the point of utter ignorance about how real A&E departments work
- There is no concrete proposal at all that defines how we will test the new targets or know care has improved
- No attempt at all has been made to use existing data to show what historic performance would look like using the new metrics (and, even worse, the single most obvious improvement in current metrics has been ignored)
I'm going to examine those in turn.
The development ignored the most relevant expert advice
The one group who have in depth expertise on the topic of how A&Es work is the Royal College of Emergency Medicine (RCEM). They were not consulted while the proposals were being developed. They are also very strongly opposed to them. This is from a letter they wrote in january 2019 before the proposals were released:
"Removing the standard will do this and hide the true scale of problems within our health service. The only ones who benefit from this are ministers and NHS managers, and certainly not patients"
"Attempting to make such change at such pace and without due regard to expert evidence is doomed to result in significant unintended consequences. The key issues to be addressed are the systemic ones to increase funding in acute beds, community care and staffing to help make our departments less crowded and improve safety. Moving the goalposts of measurement to make things seemingly look better is certainly not the way forward."
I couldn't put it better myself.
Previous work has been ignored
This point reinforces the first one. NHSE sometimes seem to have the institutional memory of a goldfish. We had a well-developed and well-supported set of new metrics to publish additionally alongside the 4hr target in 2010, authored by the then A&E Tzar Matthew Cooke. The rationale of those was to supplement the target with other indicators that would sharpen the drive to improve and constrain some of the potential of the 4hr target to drive gaming rather than improvement. There was no talk of replacing the 4hr standard. The incoming coalition government abandoned them before they were implemented.
And they seem to have been erased from history. I only recovered the document describing them from the national archives.
What impressed me in reminding myself of that 10-year old work was how much more coherent it is than the new proposals. The standards are clearer, they are focussed on ways to encourage further improvement and the evidence base is far better.
Some of the indicators proposed in the older plan are similar to some of the new indicators (but the evidence and rationale in the Cooke proposals is uniformly better than the shonky ones in the current proposals). And Cooke proposed using the new metrics alongside the 4hr target not instead of it. [amazingly, I discovered after this was originally written that the Cooke metrics are accessible but are almost completely ignored even though many would be relevant to the new plans]
The new proposals don't even reference the older ones.
The evidence is weak
One might suppose (as the RCEM seems to believe) that the primary purpose of the new proposals isn't to improve the patient experience but to minimise the number of bad headlines resulting from the monthly publication of the performance numbers. This idea explains the content better than the evidence that is used to support the metrics.
One or two of the rationales for the proposals have some merit. But they point towards supplementing the target with wider metrics not replacing it.
Some of the other arguments are shoddy to the point of embarrassing. For example one is that "patients themselves do not identify total time in department as a priority" and "the public are most concerned with time to be seen, and want to know that the sickest patients are prioritised". This is backed by public surveys conducted by Healthwatch. The problem with these views is that patients are not experts in what works inside an A&E department and driving metrics by weak patient survey results is not remotely the same as understanding what A&Es have to do to achieve results that are acceptable to the public. One problem is illustrated by this observation: no public survey on what people want from air travel puts safety in the top 3 concerns. The reason is that most airlines are very, very safe. When all airlines do well on safety, it isn't a factor that distinguishes them or achieves much share of public concern. As soon as that changes, safety rapidly rises to the top of the list and anyone not paying attention faces catastrophic business risk (as Boeing is seeing now with the software problems in its 737 Max). Let's see whether waiting times rise to the top of patient concerns as they continue to deteriorate (they were a big concern in the late 1990s and early 2000s before the 4hr target was introduced).
Even now the majority of patients leave A&E within 4hr. So very long waits are not yet a national concern for most. But the other observation from the Healthwatch survey is a clear indication that the guidelines have been written by people who don't know how A&E works. Yes, the public thinks that those with severe, urgent conditions should be "prioritised". But neither the public nor the guidelines' authors seem to understand that the best way to achieve that is not to slow down the treatment of the less sick (something that seems to have escaped both the last two health secretaries and Simon Stevens who have all said this in public). The volume of activity in A&E is dominated by minors. If they are treated more slowly, the queue in the A&E will be much bigger, impeding all the care in the department. The 4hr target was originally met by recognising that the best way to resolve the problem was to stream the flow inside A&E so that minor patients flow to a highly efficient "see and treat" process that gets them out quickly and keeps the queue small. Sufficient capacity is reserved for the less frequent majors who need more treatment that starts quickly. Streaming, not prioritisation is the key, a lesson the new proposals have forgotten and nobody expects the public to understand (unless we are now proposing universal teaching of queuing theory in primary schools). Using public opinion to support bad operational ideas is harmful and embarrassing.
As is this rationale: "the current standard measures a single point in often very complex patient pathways". I agree. But that would be no reason to abolish the measurement of mortality rates because death represents just one point in an often very complex life.
Some of the thinking in the report seems to be naive to the point of utter ignorance about how real A&E departments work
I've already mentioned the foolishness of assuming that treating minor more slowly makes it easier to treat more serious conditions faster. But the new proposals seem militantly naive on this as were several of the comments by system leaders before the proposals were published.
Not content with that misunderstanding the proposals make another howler because they haven't looked at the data clearly enough. "Hospitals’ processes, rather than clinical judgement, are resulting in admissions or discharge in the period before breach – By moving to a mean, the threshold effect is removed, allowing clinicians to admit when appropriate, rather than at an arbitrary point in time."
While it is true that the data on admission/discharge times shows some evidence of gaming the rationale that we should, therefore, use a completely different target is weak (it is really easy to highlight the degree of gaming by using supplementary metrics as well as the 4hr one as I explained here). Yes too many patients are admitted or discharged in the 15mins before 4hrs is reached. But the dominant problem in A&Es right now isn't gaming, it is long waits for admission. In most departments the majority of >4hr waits are those needing a bed. There is little opportunity to do quick admissions to avoid a 4hr breach because the hospitals usually lack the free beds to achieve quick admissions. And when there is a rush to admit, it isn't because the patient's treatment has been rushed, it is because the patient has been waiting for a bed and the admission process has been expedited. The admissions happening just before the 4hr line are not happening because treatment is being cut short, they are happening because the wait for a bed is being expedited. The proposals fall into the trap of assuming that A&E treatment dominates the time patients spend in A&E when the reality is that delays are dominated by problems of flow in other parts of the hospital. As long as the metrics make this assumption they won't incentivise the whole hospital to improve (as I've explained repeatedly before, A&E performance is not an A&E problem)
There is a significant risk that the new metrics will fossilise naive views about the problems in A&E and will lower the incentive for the whole hospital to improve flow (the target was never an A&E target, it was always a hospital target something that the Cooke proposals were very clear about but the new ones are not).
There is no clear proposal for measuring the success of the new metrics
What can I say? There is no indication whatever about how the trials of the new metrics will be assessed. And the abolition of the existing matric will make it harder to compare performance over time.
Historic data showing the performance on the new metrics has been ignored.
Most of the proposed metrics are already accessible to hospitals and national bodies with access to SUS/HES data. So the proposals could have showed us what performance would look like historically if we reported on both the new and the current metrics. This might have led to some useful refinement of the metrics or even some better evidence for where to set them. The Cooke proposals did that. The current proposals didn't.
The current proposals also claim to want to maximise the incentive for trusts to cut down long waits by using a total time in A&E metric (or the mean time for all patients). This isn't a bad idea as a supplementary metric but the report completely ignores a simple tweak to current published metrics that could radically improve our understanding of exceedingly long waits right now. Scotland, Wales and Northern Ireland all report the number of 12hr waits (end to end) for their A&E systems. England insists on reporting the utterly corrupted 12hr trolley wait target (which measures waits from the decision to admit not the time of arrival). This target is useless as the decision to admit can be postponed until a bed is available even if the patient has already waited far more than 4hr. So the number of reported 12hr trolley waits is ~2,000 in a year but the number of end-to-end waits exceeded 330,000 last year (the number isn't routinely reported and is only published at all because NHS Digital gave up trying to fight off FOI requests to extract it). It doesn't exactly promote confidence in the new targets when a simple tweak to the existing published metrics (giving us a much better insight into long waits) is ignored (perhaps because it would make some very negative headlines about how bad long waits in A&E have become).
So what?
My overall impression of the actual proposals suggests they were written by people who don't understand how A&E works (which is unsurprising given they they didn't consult the RCEM–the biggest group who do know how A&E works).
There is also very little in the proposals to dispel the idea that the ulterior motive isn't just to reduce the number of bad headlines about A&E performance. We have been here before with the unwarranted switch to monthly reporting (I explained why this was a bad idea here).
A better approach would have been to publish additional metrics to sharpen the blunt tool of the 4hr target while retaining the current metric. At least then we would have some way of tracking historic performance versus current performance. The current proposals miss obvious improvements in reporting that could be implemented now (like honest 12hr waits), ignore previous work on what other metrics could be published and fail to offer any guidance on how to judge the effect of trial rollout.
As an analyst who has worked with A&E data I also wonder why there doesn't seem to be any plan to publish the historic performance using the new metrics either nationally or–more usefully–for all the hospitals across England. All the metrics could be derived from existing national returns or the routine patient-level data collected locally. If we had several years of what performance looked like using both the new metrics and the 4hr metric we would be a lot more confident that the new ones were measuring something useful. NHSI has all the historic data. If they can't work out how to do that, I'm available for a reasonable day rate.
Overall, though, we simply can't be confident that the new proposals will drive any improvement in A&E quality or performance. But it looks as though they will deaden public discussion about the manifest problems in our emergency departments. That isn't progress.
No comments:
Post a Comment