How busy are English GPs?

Our GPs are probably grossly overworked. There is plenty of evidence that this is true, but recent data collected by NHS Digital paints an ambiguous picture. Many GPs have reacted to my analysis of that data with incredulity, claiming NHS Digital don't know what they are doing. Or that the data is meaningless and too much of a burden to collect. The real situation is more complex and won't be fixed by NHS Digital alone but requires GPs to pay more attention to how they collect data and why it is collected. And it is critically important that the system does a better job of collecting data about their activity or the case for higher primary care funding will fail as soon as it is examined by the treasury. GPs, their system providers and NHS Digital all need to work together to create more useful data.

Recently released data poses some interesting questions for GPs

At the start of 2019, NHS Digital released the first public version of a new dataset on GP activity they had been collecting since the end of 2017 (and they are currently updating monthly). This dataset summarizes the number of consultations each day using data collected directly from the (major) providers of GP clinical systems (not all of the minor providers are covered yet). 

The public version of the dataset covers the CCG-level aggregate number of appointments and the status of those appointments in several different categories:

• The appointment mode (face to face, telephone, visit, video etc.)
• The appointment status (basically whether the patient turned up for a booked slot or not)
• The type of staff who saw the patient (essentially whether the person who saw the patient was a GP or someone else)
• The delay between the booking and when the appointment actually happened

The data coverage is good for most CCGs (only one has no data at all) and there are some big, interesting insights in the data. For example, the number of no-shows ("did not attend" or DNAs) is very strongly related to the length of time a patient has to wait for an appointment (see my analysis here and a discussion in the BMJ here including comments on why the results are different to some previous academic analyses of DNA causes).

But the analysis I did on the number of appointments per day done by GPs caused a much bigger kerfuffle. I want to be fair and present those criticisms alongside the analysis. It is possible that both the analysis and GPs criticisms of it are right, but, if that is correct, then GPs, their systems providers and NHS Digital need to make some significant changes to how they work.

Before we start, a thought experiment

To put the results in context, it is worth doing a simple thought experiment to put the results in perspective.

Imagine a GP who does nothing other than see patients. She fills her entire 8-hour working day with 10-minute appointments with no breaks. Let's also assume that she works 48 weeks out of 52 (including bank holidays in the 4 weeks of downtime). And let's assume that each day worked consists of 48 appointments (this is how many fit into an 8-hour day if she takes no breaks between appointments or for lunch and does any other work after hours or at weekends). That means she does 11,520 appointments per year or about 44 per weekday in the year (she doesn't work every weekday because of holidays).

That 44 appointments per day doesn't sound like a sustainable workload. So let's assume a 1hr lunch break and gaps between appointments that brings the average work done down to 5 appointments per hour. Now she does 8,400 appointments per year or about 32 per weekday averaged over the year. That's probably still not sustainable every year but it is a thought experiment and we can make the simplifying assumption that GPs are superhuman. Bear this number in mind when we look at the actual reported workload from the NHS Digital data.

And, don't forget, that a real GP also has a ton of paperwork to process plus they need time for professional development, time to train new GPs and other staff and time to manage the practice. So lots of other work to fit into the actual working day alongside the 8-hr shift spent actually doing appointments. And, remember this analysis simply takes the total number of appointments recorded in a CCG divided by the number of weekdays and the number of FTE GPs; it ignores all the other work and all the extra time GPs put in beyond normal hours or at weekends.

What does the NHS Digital data show?

The original claim that prompted me to look more closely at the NHS Digital data was a simple calculation about the number of appointments per GP based on the headline numbers. NHS Digital estimate that about 307m appointments were recorded in the year to April (they adjust the total for missing data and their adjustment seems to differ from mine by about 10m/year but this is not a huge gap given other uncertainties in the data). Taking their number per working day in the year and adding the March estimate that there are 33,425 FTE GPs in England suggests that the average GP does about 35 appointments per working day. This seems to confirm, when compared to my thought experiment above, that the typical workload is well into unsustainable territory with a side order of "if we keep this up we are all going to die of overwork".

But that isn't what the dataset actually says. GPs have a lot of other staff who handle >40% of the patient contacts according to the data (this is the national average but see the map below for the variation among different CCGs in the % seen by non-GPs).

If you take that into account, the data actually says the average GP does something closer to 20 appointments per day. 

20 appointments/day is the England average. But, usefully, we have the staff census data that tells us how many FTE GPs there are in each CCG and the appointment counts are available for each CCG as well. So I can do a CCG-level analysis and look at the variation across the country (taking into account the number of GPs in the CCG and the % of appointments with a GP). If we generously assume that activity where the staff type for the staff fulfilling the appointment is "unknown" is done by GPs we get the following distribution:

(Note: each block on this chart is a single CCG and the activity data is based on activity done on weekdays in the first 4 months of 2019. Also note that an interactive version of the dataset and many analyses is available here and you can test how varying some of my assumptions changes the results.)

In about 8 CCGs the GPs appear to be seeing an average of about 26-30 patients per day. The commonest CCG average is, however, closer to 20/day. Also, don't forget, the results are averaged over at least a month and reflect the average activity on all available days, not just the busy days where all the GP time is devoted to seeing patients.

Don't forget, though, that this is the average across a whole CCG: the variation among practices within each CCG is probably large so there will probably be some practices where the GPs do >30/day even when the CCG average is just 20.

Some GPs were indignant when I put an early version of this chart on Twitter. The typical response was "20 a day! I do far more work than that in a typical day. NHS Digital don't know what they are doing. The data is obviously rubbish…". I'm not entirely convinced that all the criticisms are fair, but it is worth looking at some of them to see if there are lessons to be learned.

Possible problems with the data

This just doesn't reflect my typical day

This could be because the data is just wrong, but we will come to that possibility later.

Another possible explanation is that what GPs remember about how busy they are isn't an accurate reflection of the average number of appointments across the whole month or year. If, for example, what GPs remember are the day when they are focussed on seeing patients but they also devote other days to practice management, paperwork, training or some other work that doesn't involve booked appointments, then their memory won't match the averages reported here: the results will reflect the total appointments divided by the total available time for all work not just the typical day when the GP is dedicated to seeing patients. 

This work (and my initial thought experiment) assumes a 40hr week. Many GPs spend a lot more than 8hrs a day working (perhaps doing all the admin in their unpaid overtime). This dataset is not recording that work or adjusting the available hours to account for it. This doesn't impact the results but it may make the perception of the results much less clear than they should be.

The data clearly is missing a lot of the work we do

Part of this is a perception problem as discussed above. But it could clearly be true that a lot of activity is not correctly recorded. 

But the blame here isn't on NHS Digital. The data is directly collected from the major clinical systems used in practices to record activity. If the numbers are a big understatement of the appointments being booked, then practices have a big problem not just NHS Digital. I suspect we are not understating the true number of appointments by a large margin.

But we could be understanding the amount of activity. Many patient contacts don't result in a booked face-to-face appointment. GPs do some work online or by telephone. Clinical systems are not uniformly good at recording that activity as I know from comparing incoming requests via askmyGP (an online tool that manages all incoming patient requests and helps GPs to completely change their workflow when responding to those requests in ways that improve the speed of responses to patients and lowers the degree of overwork for GPs). Successful online tools (and some phone-triage approaches) often result in only 1 in 3 requests leading to a face-to-face appointment. This reduces the number of appointments recorded in the clinical system while providing satisfactory responses to the other patients. But much of the activity for those other responses may never appear in the clinical systems. As GPs adopt new ways of working their clinical systems may become a lot less good at measuring their activity.

On the other hand, the NHS Digital data does record a lot of phone activity. Between 10 and 15% of all appointments are recorded as taking place online, by phone or by video. We know that recording of this is patchy, but so is uptake of alternative ways of responding to patient demand. Some GPs still think that face-to-face appointments are the only response despite mounting evidence that offering a wider range of alternatives can reduce GP workload and make patients happier because they get a faster response.

We have to admit, though, that traditional GP systems have not caught up with this change in practice and won't consistently record the shift in activity reliably.

NHS Digital will draw all the wrong conclusions from these results

I got the impression from some comments on my initial analysis that, not too exaggerate too much, central NHS bodies are just a bunch of malingering bureaucrats who exist just to malign hard working GPs by misrepresenting how hard they work by providing the NHS leadership with false, irrelevant data about what they do. And that any suggestion that this should be fixed by GPs spending more time to provide accurate data about what they do would be yet another straw added to the already broken camel's back that is the typical GPs unsustainable workload (or some such mixed metaphor for even more overwork).

But NHS Digital's motivation is not to burden GPs with more work or to undermine the case for more GPs. It is to understand what GPs are doing to develop better policy, and that can't be done without better data. Originally the idea was focussed on developing a better understanding of winter pressures but it should be useful for bolstering the case for more investment in primary care or for promoting better working processes. 

But, imagine the conversation the NHS would have with the treasury when bidding for the money to recruit another 5,000 GPs:

NHS "We need another 5,000 GPs."

HMRC "How much will that cost?"

NHS "A couple of billion pounds a year."

HMRC "That sounds like a lot. Why do you need it?"

NHS "They keep telling us they are overworked."

HMRC "That's what everyone says. How much work do they do? Show me some numbers."

NHS "They say they are doing unsustainable numbers of appointments a day."

HMRC "How many is that."

NHS "We don't know exactly but more than 30/day doesn't sound sustainable"

HMRC "So how many are they actually doing?"

NHS "No idea. But I'm sure it is a lot. We really need the money to recruit more GPs."

HMRC "Sure. You can have it when we cash the £350m/week we will save from Brexit."

NHS "Really?"

HMRC "No." 

HMRC "We were joking. Sod off and don't come back until you have some actual evidence."

OK, I'm parodying the case for more money for GPs and there are other hard sources of evidence that say they are overworked. But if the NHS as a whole doesn't understand what, in detail, GPs are actually doing any case for reform or extra cash is going to look weak.

There are more reasons than this to have reliable data. GPs need it to understand their own work or they won't have any idea how to do a better job. While the NHS Digital appointment data doesn't capture everything they do, it is at least a start providing a better understanding of a large part of their activity. And the main limitations that prevent the data being more useful are not NHS Digital's fault but the fault of the system providers and choices made by GP practices. 

The data isn't consistent or comprehensive

GP Clinical systems are built around a model of GP work that assumes everything revolves around a pre-booked face-to-face appointment. GPs can vary how they label the slots for activity in the system and they don't always assume every appointment is 10mins long (though their royal college doesn't seem to have caught up with this realisation). Systems usually allow the exact time taken for each appointment to be recorded (assuming GPs press the right buttons which they don't always do). Some systems make it easy to record activity in open-access clinics. But systems tend to assume every appointment slot is the same length and some are completely, irretrievably hopeless at measuring the time taken for consultations done by phone. 

One of the biggest problems faced by NHS Digital is the number of different descriptions given as labels for each slot. Practices are, essentially, free to label each slot however they want and one estimate says that there are tens of thousands of different labels in use across practices in England. So when NHS Digital try to acquire data from everyone it is a gargantuan task to correctly identify the different types of appointment and group them together in ways that make sense. Was that a face to face pre-booked appointment? Or an ad-hoc on the day f-to-f as part of a walk-in clinic. Or a slot where a nurse did flu-jabs?

And, as more activity is initiated online and GPs choose to triage patient requests before booking appointments, there are more varied ways for GPs to respond to patients. And the systems–designed as they are around the idea that most activity involves pre-booked slots–don't record all those interactions. In practices using online tools like askmyGP only around ⅓ of requests result in a face-to-face appointment (e-Consult, another online tool provider estimates a similar ratio). This means, potentially, that the majority of GP activity won't be recorded in the traditional clinical systems.

So it may well be true that what NHS Digital are collecting is not a reliable guide to what activity GPs are doing. But the fault can't be fixed by them: if we want better data the problem has to be addressed by the practices (who need more consistency is how they use their systems) and the suppliers (who need to design their systems to break the model that everything neatly fits into 10 or 15min slots).

Some suggestions about how to do a better job

It is not at all helpful to complain that the NHS Digital GP data is rubbish. If we want to do a better job in primary care (to improve how GPs work and make an unanswerable case for more investment) we need better data. 

An article in the Economist describes some improvements in primary care and how they were done:

When the four practices serving St Austell merged in 2015, it was an opportunity to reconsider how they did things. The GPs kept a diary, noting precisely what they got up to during the day. It turned out that lots could be done by others: administrators could take care of some communication with hospitals, physios could see people with bad backs and psychiatric nurses those with anxiety. So now they do. Only patients with the most complicated or urgent problems make it to a doctor.

The key point here is that only when they had reliable data about their activity could the GPs redesign the work they did in ways that would improve it.

As a recent Nuffield Trust comment on the GP dataset says:

In hospitals, routine datasets such as Hospital Episode Statistics (HES) give us insight into who gets treated, for what, by whom, and where. This basic knowledge of the activities and performance of hospitals is now essential for making evidence-based policy changes, monitoring existing policies and providing fundamental administrative data to run the system and allocate funds. There is a clear need for something similar in general practice…

But, they continue:

NHS Digital makes it clear that what has been released so far still needs improvement. As it stands, there are a number of problems with the data and these limit the extent to which we can use it as a resource for evidence-based policy. 

Having different practice IT systems gives GPs flexibility over the way they organise their work. But the absence of reporting standards in general practice means information is recorded differently in each system, making it difficult to combine the data, and information gets lost.

In the early 2000s, hospital activity data (HES) was in a similar state of disorder. Doctors took little interest in making sure that the amount of activity recorded was correct or the clinical details correctly coded. They too argued that this was just an unnecessary bureaucratic task required puerly for "feeding the machine" (even though the most basic metric of hospital success–not killing your patients too often–requires accurate coding). 

But HES got a lot better. One way it got better was to show medics what HES said about the work they did. The Royal College of Physicians ran some interesting experiments starting in 2002 (reported here and here). They took HES data and fed it back it to the clinicians whose activity it recorded. In one of the reports on this exercise they argue this:

A vicious circle ensues: routinely collected data is perceived as being of poor quality and unable to support the needs of the individual. Individual clinicians avoid the use of such readily available information... Centrally held datasets remain unchanged through neglect, clinicians failing to engage with the information process in their trusts and remaining ill at ease with the records of activity which result. It is clear that if this cycle is to be broken, steps must be taken to engage clinicians at a level whereby the information is made readily available, accessible in format and of use to clinical practice. By examining routine data from a clinical perspective and feeding issues of quality back to trust information departments the cycle can be reversed.

Hospital clinicians being shown their own data in a digestible form was a big part of the drive that improved the quality and utility of HES data. GPs and NHS Digital need to do something similar.

Perhaps NHS Digital could take the first step by making practice level data from the central collection available to practices in a useful, comprehensible format so GPs can see what the data actually thinks they did. This would be the first step in driving improvement in what is recorded.

Conclusion

In short, here are some simple ways for the different players in primary care to change what they do in ways that would promote real improvement for GPs and their patients.

NHS Digital should make a friendly, easy to comprehend, version of their dataset available to every individual practice so GPs can compare what they think they did to what NHS Digital's extract has recorded.

GP System Providers should make their systems more flexible in recording non-appointment activity and should work to support GPs to use more consistency in how they record and label their activity.

GPs and practices should not just dismiss the data as being unreliable and useless. They should strive to understand it and seek ways to work with NHS Digital to make the data useful for local purposes and more reliable for policy making.

Everyone should not dismiss data they don't like but engage with it and seek to make it both more reliable and more useful.

Yes, the NHS has big problems but stupid populist headlines won't fix them

Several popular news stories in the last couple of weeks have shown that attention-grabbing headlines do more to distract than they do to improve the NHS. You have to know what the problem is before you fix it and too many popular discussions on the system's problems identify the wrong problem and get in the way of actual improvement.

If your idea of what is wrong with the NHS is based on reading popular news stories, then I have bad news for you. Headlines are, more often than not, driven by lobbyists with an agenda or journalists looking for link-bait and not by people who are really concerned to fix anything. In some cases they are simply pushing a partisan agenda with little regard for the facts about where the real failings are; in other cases they are deliberately trying to distract you from the real problem. 

Here are three examples:

• NHS leaders have recently made speeches about the malign impact of the anti-vaccination movement
• Lobbying group CHPI have claimed that NHS cuts are driving people to self-fund cures the NHS won't pay for
• The Royal College for GPs (RCGP) have complained that 10 min GP appointments are inadequate for patient needs

Here is what is wrong with those campaigns and the headlines they generated.

It is the NHS not the anti-vaccination movement that is to blame for low vaccination rates

In the last couple of months both Simon Stevens (the boss of NHS England) and Matt Hancock have publicly complained about the influence of the anti-vaccination movement on vaccination rates in the UK. There have even been well received suggestions that the government should introduce compulsory vaccination as a way to combat the trend. Surely all right-thinking, well informed, people should agree that low vaccination rates are a problem and that the government should do more to combat the malign influence of the scientifically illiterate anti-vaccination movement?

Here is the problem: the cause of low vaccination rates in the UK isn't the strong social media influence of the anti-vaccination movement. The real problem is inside the NHS where the processes that should ensure vaccinations are carried out are poorly managed and coordinated. This blog from the Science Media Centre quotes some of the experts:

Research by the Royal Society of Public Health seems to support this suggesting that drops in vaccination levels may be due to timing, availability and location of appointments. And Dr Mary Ramsay from Public Health England has said they believe that sending out reminders and making GP appointments as convenient as possible will make ‘the biggest difference’ in reducing numbers of unvaccinated children.

It points out that the best evidence we have suggests that the influence if the malign anti-vaccination movement is weak. The uptake of the first dose of the two-stage MMR vaccine for example, is much higher than the uptake of the second dose which would not be true if parents objected to the vaccine but is far more likely caused because of poor reminders to have the second dose or poor availability of clinics or appointments that offer it.

In the pre-Lansley NHS, PCTs employed vaccine coordinators to organise campaigns and make sure vaccinations happened. Those people were mostly culled in the orgy of management cuts when CCGs were created (which were supposed to direct "more resources to the front line"). But, as I've argued before, this slogan is as dumb as a box of spanners. The NHS was undermanaged before those management cuts and cutting management further has just made the actual work done in healthcare less coordinated and less effective.

In short, the NHS leadership's attempt to blame low vaccination rates is fake news designed to distract us from the real problems caused by foolish historic NHS policy and current NHS failings.

Popular campaigns to fund treatments the NHS won't pay for are nothing to do with a lack of NHS resources

A recent tweet from the CHPI said the following:

The BBC reports that patients unable to access NHS care have raised £20 million through crowd-funding to pay for care privately. As NHS underfunding continues it is sadly inevitable that more patients will have to pay out of pocket to get healthcare

It referenced this BBC story about parents who had raised money to fund a cancer cure the NHS wouldn't pay for. The BBC story was somewhat more balanced than the CHPI tweet, noting the family's doubts they had done the right thing and that the cure had, ultimately, failed.

But while children dying of cancer and the NHS denying them treatment make good news headlines, they tell us nothing useful about NHS funding or policy. Even an NHS with an unlimited budget should not be paying for dubious, untested "cures" which, most likely, won't work. If parents want to crowdfund large sums of money to buy themselves false hope, fine. But no health service that cares about using evidence should have any obligation to fund those false hopes.

It is bad enough that the NHS has–against the advice of almost every health economist who can count–wasted billions on the Cancer Drugs Fund (designed, essentially, to pay for expensive treatments of dubious efficacy that have been ruled out by NICE, the body that evaluates the costs and benefits of new medicines). The fund's real purpose is to minimise the number of bad headlines. No health system that takes evidence seriously should have any obligation to pay large sums for treatments that, most likely, won't work. And linking such decisions to a shortage of money in the system is purely a distraction from the hard choices any health system–however well funded–has to make.

The length of GP appointments is not the government's problem

Many groups in the NHS like to deploy any evidence they can lay their hands on to lobby for a bigger slice of the pie for their part of the system. And the general argument that more resources would help is often true. But, sometimes, the argument a major distraction and gets in the way of effective solutions. So it is with the recent argument that the 10 minute GP appointment is "inadequate".

The RCGP argument is roughly this:

The 10-minute appointment that is usual across the NHS is no longer adequate given the growing number of people who have several long-term illnesses, according to a report from the Royal College of GPs. 

However, while longer consultations are needed to ensure proper patient care, they will only be possible if the staffing of GP surgeries increases significantly, the college admits.

Or, more simply, patients need more time so we need more GPs.

The second part of this is probably true: we do need more GPs.

It is the first part that is a problem. For a start, there is no obligation for all appointments to take 10 minutes or to be conducted face-to-face. GPs are free to choose a mix of flexible consultation lengths or to do consultations online or by phone. Some do. But many others don't, not least because their professional body keeps complaining about the idea that all appointments should consist of a 10 minute face-to-face meeting in its lobbying campaigns.

Secondly, the self-imposed inflexibility of 10 minute slots being the primary way a GP interacts with patients is a major cause of GP overload. GPs who break this assumption have found they can do a much better job for their patients while controlling their workload. Evidence from online consultation tools like askmyGP (COI: I provide their analytics) suggest that perhaps ⅔ patients wanting help from their GP don't need a 10 minute appointment. Some can get a satisfactory response online or by telephone. When GPs triage the patient need before offering an appointment, far fewer patients need to visit the GP and the time saved by the GP can be used to offer more flexible slots for those who do need to be seen. See the video in this tweet for a story from a GP practice who have experience this change.

In short, a better-designed, more flexible process for responding to patients delivers a better service for patients and lower stress for the GP. None of this solves the overall shortage of GPs but it can make it easier to solve by lowering the stress and increasing the attractiveness of the work making it easier to retain and recruit staff.

But every time the RCGP talk about 10 minute appointments as a basis for lobbying for more GPs, they distract from such solutions by reinforcing the inflexible view that the only way to serve patients is to offer inflexible face-to-face slots. Again, the easy headline damages the ability to seek better solutions to the problem.

So what? Don't be fooled by headlines that distract you from the real issues

The message is simple: what looks good in headlines often gets in the way of your understanding of the real problem. Good analysis of what is broken in the NHS rarely makes a good headline; nonsense that distracts from the real problem often does. Don't make policy from headlines; apply skeptical analysis; don't be distracted by fake news.

Publishing wider metrics about A&E performance is a good idea: abandoning the 4hr target is not

The current proposal by NHSE to replace the 4hr A&E target is muddled and confusing. It isn't strongly supported by evidence. It will, most likely, make the experience of patients in A&E even worse. Worse still, if things do get worse we won't have the available public data to measure the deterioration. If they paid any attention to history or the experts in emergency medicine, they could have done a much better job.

What is in the proposals?

As part of a review of NHS Access Standards NHSE propose replacing the current A&E target (95% of all patients to leave A&E inside 4hr) with a set of new targets. Those new targets are:

• Time to initial assessment
• Time to treatment for conditions requiring urgent treatment
• Mean time spent by all patients in A&E
• Utilisation of same-day emergency care

It is a little unclear what the last one of these means, but it looks like it will be based on some metric like the proportion of admissions with a zero length of stay (which the report seems to want to encourage as a way of avoiding overnight stays from admission).

What is the rationale for this?

The claimed rationale is to "measure what's most important clinically, and to patients". In support of this, the review suggests the current standard fails on several important grounds:

• It doesn't measure total waiting time
• It doesn't differentiate between severity of condition
• It measures a single point on a complex pathway
• Hospitals' processes, rather than clinical judgement are rushing discharge or admission decisions
• The standard is not well understood by the public

The idea being, at least partially, that the new standards will "drive better outcomes", drive improvement and reduce the opportunity for gaming of current targets.

All that sounds reasonable so what is the problem?

There are multiple problems with the new proposals and they fall into several categories:

• The development ignored the most relevant expert advice
• Previous work (which did use input from experts who knew their stuff) has been ignored
• The evidence used to support the new proposals is weak
• Some of the thinking in the report seems to be naive to the point of utter ignorance about how real A&E departments work
• There is no concrete proposal at all that defines how we will test the new targets or know care has improved
• No attempt at all has been made to use existing data to show what historic performance would look like using the new metrics (and, even worse, the single most obvious improvement in current metrics has been ignored)

I'm going to examine those in turn.

The development ignored the most relevant expert advice

The one group who have in depth expertise on the topic of how A&Es work is the Royal College of Emergency Medicine (RCEM). They were not consulted while the proposals were being developed. They are also very strongly opposed to them. This is from a letter they wrote in january 2019 before the proposals were released:

"Removing the standard will do this and hide the true scale of problems within our health service. The only ones who benefit from this are ministers and NHS managers, and certainly not patients"

This from a statement in march (my highlighting):

"Attempting to make such change at such pace and without due regard to expert evidence is doomed to result in significant unintended consequences. The key issues to be addressed are the systemic ones to increase funding in acute beds, community care and staffing to help make our departments less crowded and improve safety. Moving the goalposts of measurement to make things seemingly look better is certainly not the way forward."

I couldn't put it better myself.

Previous work has been ignored

This point reinforces the first one. NHSE sometimes seem to have the institutional memory of a goldfish. We had a well-developed and well-supported set of new metrics to publish additionally alongside the 4hr target in 2010, authored by the then A&E Tzar Matthew Cooke. The rationale of those was to supplement the target with other indicators that would sharpen the drive to improve and constrain some of the potential of the 4hr target to drive gaming rather than improvement. There was no talk of replacing the 4hr standard. The incoming coalition government abandoned them before they were implemented.

And they seem to have been erased from history. I only recovered the document describing them  from the national archives.

What impressed me in reminding myself of that 10-year old work was how much more coherent it is than the new proposals. The standards are clearer, they are focussed on ways to encourage further improvement and the evidence base is far better.

Some of the indicators proposed in the older plan are similar to some of the new indicators (but the evidence and rationale in the Cooke proposals is uniformly better than the shonky ones in the current proposals). And Cooke proposed using the new metrics alongside the 4hr target not instead of it. [amazingly, I discovered after this was originally written that the Cooke metrics are accessible but are almost completely ignored even though many would be relevant to the new plans]

The new proposals don't even reference the older ones.

The evidence is weak

One might suppose (as the RCEM seems to believe) that the primary purpose of the new proposals isn't to improve the patient experience but to minimise the number of bad headlines resulting from the monthly publication of the performance numbers. This idea explains the content better than the evidence that is used to support the metrics.

One or two of the rationales for the proposals have some merit. But they point towards supplementing the target with wider metrics not replacing it.

Some of the other arguments are shoddy to the point of embarrassing. For example one is that "patients themselves do not identify total time in department as a priority" and "the public are most concerned with time to be seen, and want to know that the sickest patients are prioritised". This is backed by public surveys conducted by Healthwatch. The problem with these views is that patients are not experts in what works inside an A&E department and driving metrics by weak patient survey results is not remotely the same as understanding what A&Es have to do to achieve results that are acceptable to the public. One problem is illustrated by this observation: no public survey on what people want from air travel puts safety in the top 3 concerns. The reason is that most airlines are very, very safe. When all airlines do well on safety, it isn't a factor that distinguishes them or achieves much share of public concern. As soon as that changes, safety rapidly rises to the top of the list and anyone not paying attention faces catastrophic business risk (as Boeing is seeing now with the software problems in its 737 Max). Let's see whether waiting times rise to the top of patient concerns as they continue to deteriorate (they were a big concern in the late 1990s and early 2000s before the 4hr target was introduced).

Even now the majority of patients leave A&E within 4hr. So very long waits are not yet a national concern for most. But the other observation from the Healthwatch survey is a clear indication that the guidelines have been written by people who don't know how A&E works. Yes, the public thinks that those with severe, urgent conditions should be "prioritised". But neither the public nor the guidelines' authors seem to understand that the best way to achieve that is not to slow down the treatment of the less sick (something that seems to have escaped both the last two health secretaries and Simon Stevens who have all said this in public). The volume of activity in A&E is dominated by minors. If they are treated more slowly, the queue in the A&E will be much bigger, impeding all the care in the department. The 4hr target was originally met by recognising that the best way to resolve the problem was to stream the flow inside A&E so that minor patients flow to a highly efficient "see and treat" process that gets them out quickly and keeps the queue small. Sufficient capacity is reserved for the less frequent majors who need more treatment that starts quickly. Streaming, not prioritisation is the key, a lesson the new proposals have forgotten and nobody expects the public to understand (unless we are now proposing universal teaching of queuing theory in primary schools). Using public opinion to support bad operational ideas is harmful and embarrassing.

As is this rationale: "the current standard measures a single point in often very complex patient pathways". I agree. But that would be no reason to abolish the measurement of mortality rates because death represents just one point in an often very complex life.

Some of the thinking in the report seems to be naive to the point of utter ignorance about how real A&E departments work

I've already mentioned the foolishness of assuming that treating minor more slowly makes it easier to treat more serious conditions faster. But the new proposals seem militantly naive on this as were several of the comments by system leaders before the proposals were published.

Not content with that misunderstanding the proposals make another howler because they haven't looked at the data clearly enough. "Hospitals’ processes, rather than clinical judgement, are resulting in admissions or discharge in the period before breach – By moving to a mean, the threshold effect is removed, allowing clinicians to admit when appropriate, rather than at an arbitrary point in time."

While it is true that the data on admission/discharge times shows some evidence of gaming the rationale that we should, therefore, use a completely different target is weak (it is really easy to highlight the degree of gaming by using supplementary metrics as well as the 4hr one as I explained here). Yes too many patients are admitted or discharged in the 15mins before 4hrs is reached. But the dominant problem in A&Es right now isn't gaming, it is long waits for admission. In most departments the majority of >4hr waits are those needing a bed. There is little opportunity to do quick admissions to avoid a 4hr breach because the hospitals usually lack the free beds to achieve quick admissions. And when there is a rush to admit, it isn't because the patient's treatment has been rushed, it is because the patient has been waiting for a bed and the admission process has been expedited. The admissions happening just before the 4hr line are not happening because treatment is being cut short, they are happening because the wait for a bed is being expedited. The proposals fall into the trap of assuming that A&E treatment dominates the time patients spend in A&E when the reality is that delays are dominated by problems of flow in other parts of the hospital. As long as the metrics make this assumption they won't incentivise the whole hospital to improve (as I've explained repeatedly before, A&E performance is not an A&E problem)

There is a significant risk that the new metrics will fossilise naive views about the problems in A&E and will lower the incentive for the whole hospital to improve flow (the target was never an A&E target, it was always a hospital target something that the Cooke proposals were very clear about but the new ones are not).

There is no clear proposal for measuring the success of the new metrics

What can I say? There is no indication whatever about how the trials of the new metrics will be assessed. And the abolition of the existing matric will make it harder to compare performance over time.

Historic data showing the performance on the new metrics has been ignored.

Most of the proposed metrics are already accessible to hospitals and national bodies with access to SUS/HES data. So the proposals could have showed us what performance would look like historically if we reported on both the new and the current metrics. This might have led to some useful refinement of the metrics or even some better evidence for where to set them. The Cooke proposals did that. The current proposals didn't.

The current proposals also claim to want to maximise the incentive for trusts to cut down long waits by using a total time in A&E metric (or the mean time for all patients). This isn't a bad idea as a supplementary metric but the report completely ignores a simple tweak to current published metrics that could radically improve our understanding of exceedingly long waits right now. Scotland, Wales and Northern Ireland all report the number of 12hr waits (end to end) for their A&E systems. England insists on reporting the utterly corrupted 12hr trolley wait target (which measures waits from the decision to admit not the time of arrival). This target is useless as the decision to admit can be postponed until a bed is available even if the patient has already waited far more than 4hr. So the number of reported 12hr trolley waits is ~2,000 in a year but the number of end-to-end waits exceeded 330,000 last year (the number isn't routinely reported and is only published at all because NHS Digital gave up trying to fight off FOI requests to extract it). It doesn't exactly promote confidence in the new targets when a simple tweak to the existing published metrics (giving us a much better insight into long waits) is ignored (perhaps because it would make some very negative headlines about how bad long waits in A&E have become).

So what?

My overall impression of the actual proposals suggests they were written by people who don't understand how A&E works (which is unsurprising given they they didn't consult the RCEM–the biggest group who do know how A&E works).

There is also very little in the proposals to dispel the idea that the ulterior motive isn't just to reduce the number of bad headlines about A&E performance. We have been here before with the unwarranted switch to monthly reporting (I explained why this was a bad idea here).

A better approach would have been to publish additional metrics to sharpen the blunt tool of the 4hr target while retaining the current metric. At least then we would have some way of tracking historic performance versus current performance. The current proposals miss obvious improvements in reporting that could be implemented now (like honest 12hr waits), ignore previous work on what other metrics could be published and fail to offer any guidance on how to judge the effect of trial rollout.

As an analyst who has worked with A&E data I also wonder why there doesn't seem to be any plan to publish the historic performance using the new metrics either nationally or–more usefully–for all the hospitals across England. All the metrics could be derived from existing national returns or the routine patient-level data collected locally. If we had several years of what performance looked like using both the new metrics and the 4hr metric we would be a lot more confident that the new ones were measuring something useful. NHSI has all the historic data. If they can't work out how to do that, I'm available for a reasonable day rate.

Overall, though, we simply can't be confident that the new proposals will drive any improvement in A&E quality or performance. But it looks as though they will deaden public discussion about the manifest problems in our emergency departments. That isn't progress.