We are having the wrong debate about modelling the NHS workforce

It sounds intuitive that having a good model of the NHS workforce would be useful for solving many observable problems in the current NHS. But there are reasons to assume any such model would not be as useful as expected and could even be harmful. More importantly, the starting point suggested for the model is wrong in multiple ways that almost certainly guarantees the model would fail to solve the real problems.

When I first heard that a coalition of Labour MPs and the ex-NHS SoS Jeremy Hunt were proposing an amendment to the Health Bill to insist on regular publication of a workforce model for the NHS, I thought the idea was a good one. Better, transparent information about the staffing needs of the future NHS sound like a useful idea to test against government policy. 

But, when I reflected on some of the issues I have seen in the past when Strategic Health Authorities did workforce plans I started to have doubts. Then, after some further conversations and cogitation, those doubts grew. Considered alongside my analysis of what the biggest challenges are for the NHS (in short, the front line workforces is far from the biggest problem), my skepticism strengthened. 

So, I'm going to argue that, while a good workforce plan might help, the one we are likely to get is likely to be somewhere between useless and harmful. It is starting in the wrong place, has unclear goals that will likely make it far less useful than expected and has some risk of making things worse. 

That's a big claim. Let me walk through the potential issues I see step by step. 

Where the workforce model might go wrong

It doesn't start by considering productivity

The starting assumption in the debate is the almost universal belief that the problem is a shortage of front line staff. Commentators observe busy A&Es, overwhelmed GPs risking staff burnout, hospitals where waiting lists are growing not falling, and leap to the conclusion that the only way to address these is more medical staff.

But the problem framed this way distracts from any analysis that concludes anything other than "more staff" can influence the amount of work done. Concluding that only more staff matters rules out many known interventions that should be part of the debate, especially those that improve productivity..

Here is a simple example. NHS A&Es are currently catastrophically crowded and patients are getting treatment so slow it is killing them. But we know from analysis that has existed since the 4hr target was introduced that the number of A&E doctors has very little influence on the speed (for a fun review of the evidence on this and how little appetite the system has to listen to it read this BMJ piece and the replies). The dominant cause of long waits in the last decade has been slow access to free beds for admitted patients. The problem isn't even inside the A&E, so adding more A&E staff won't fix it. To be fair, the workload needs in A&E do increase sharply with the length of the queue. But they don't help to make the queue shorter. So any workforce model that ignores the external factors causing the queue will end up recommending far more A&E staff than needed if the external bottleneck causing the queue is ever solved.

Another example shows that not thinking more widely about the mix of staff required is a major problem in planning. A Royal College of Surgeons blog reported in 2017 that the productivity of surgeons had declined sharply as their numbers rose because numbers of support staff and nurses had not risen. It reports "Between 2010 and 2016, consultant numbers rose by 22%, compared to just 1% for nurses and 2% for all staff."  It Also pointed out that "... consultants in hospitals that invested more in infrastructure and building … are more productive". (it is worth reading the original analysis by the Health Foundation as well for much more detail). So, is a workforce model is built to forecast the number of surgeons required, but ignores the number of nurses and support staff or the capital required to create a productive theatre, it will vastly overestimate the number needed.

The point is that productivity depends on the mix of staff and other factors like equipment. Driving higher output across the NHS requires a good understanding of where the bottlenecks to productivity are so the right mix of capital and people can be deployed. Adding more of the most visible front line staff is often not that effective. But it is what a "more resources to the front line" workforce model is likely to achieve.

It is unclear what decisions a long term workforce model is intended to support

The only point of any model is to support better decisions. If you are vague about what decisions, then the model is likely to be unhelpful and even misleading. 

So what decisions could a long term workforce model support? So far the discussion has tended to focus on the need for a model to inform the NHS about its workforce need in 5, 10 or 15 years. 

What decisions could such a model influence? Not many. If we know we need more nurses in 5 years time the NHS might just have enough time to increase the number of training places to increase the numbers qualifying in 5 years time. But it is unclear whether increasing the number of doctors in training right now would lead to higher available numbers in a decade's time. 

If the NHS is short of particular skills right now, it is unclear how a long term model can possibly help. What the system needs most urgently is some idea of what the options are today. 

If the model focuses–as much of the discussion about it has–on front line staffing need, then it also misses critical groups that contribute to the productivity of the front line (see the section on productivity for why this is important). The question that desperately needs an answer is what different mix of staff, equipment, buildings and new clinical processes would give the biggest increase in the number and quality of treatments delivered. Many of those questions are not workforce questions at all and even a workforce-only model needs a good understanding of how different staff interact to make the front line more productive. That understanding will only come from a significant piece of careful analysis that doesn't seem to exist. 

What the NHS needs right now is that analysis. It needs to know where the bottlenecks to higher activity are. It needs to know what mix of capital spending, front line staff, support staff and managers would lead to the largest improvement. Without that a workforce plan will be about as useful as a one-legged trapeze artist with an itchy bum.

A workforce model designed to tell the system how many staff it needs to put into training now will get the wrong answer because it ignores all the interactions and other factors that matter and, even if that wasn't true, could not influence any decision that will have an effect for 5-10 years at best.

Major factors that influence the workforce today are likely not part of the model

The NHS has a workforce problem right now. And many of the factors causing problems are not relevant to the long term supply of qualified staff. Or anything else likely to appear in the proposed workforce model.

Right now the biggest factors influencing staffing gaps are recruitment problems and high turnover (plus illness, if temporary pandemic-specific problems count). These problems don't just affect the front line staff groups but are common in the other groups where a lack of staff has a lot of leverage over front line productivity.

There are many causes of recruitment problems and high turnover. In some groups NHS pay is inadequate compared to other jobs the same staff can do. This is a big issue for nurses but a huge problem for support staff like data scientists. It is also a bigger problem in some geographies like London where the cost of living is much higher and there are more alternative well-paid jobs. But the NHS finds it almost impossible to flex salaries to retain the people it needs both because the pay scales are national and because some groups are vastly undervalued in AfC grading compared to the market.

Working conditions are also a huge factor for recruitment and turnover. If the space is badly adapted to the work being done (~14% of buildings predate the NHS!) then the environment will be poor. Badly maintained buildings add to this (the maintenance backlog is about £10bn). Old, shonky equipment is slower and harder to use than modern equipment. IT systems are often slow and not seamlessly integrated so staff waste time waiting to log on or logging in to a dozen separate systems to complete a clinical task. Front line staff end up spending too much time doing tasks that should be done by support staff or managers (where staffing levels have been cut to "put more staff on the front line") instead of caring for patients. 

Too much of the people management in the NHS is bad. Staff are treated badly and insensitively by managers but also by senior doctors and nurses (the Ockenden report didn't just blame "staff shortages", it clearly blamed senior staff of all professions for ignoring clear signals about problems and even suppressing whistleblowers). 

Very few, if any, of the factors that discourage recruitment and drive high turnover are part of any proposed workforce model.

So the model won't tell the NHS whether a big increase in capital spending, creating better buildings, equipment and IT systems, would yield rapid gains in a better working environment. Nor will it conclude that recruiting more support staff to enable the front line to focus on treating, rather than admin paperwork, would improve their job satisfaction. And nobody in NHSE would allow the model to conclude that salary flexibility might yield immediate benefits in both lower turnover and higher recruitment rates.

That means that the model is likely to have nothing to say about the major factors that could impact the workforce any time in the next 5 years. What was the point of it again?

A long term workforce model risks fossilising current mistakes and practices

More than a decade ago I was part of a team auditing some workforce models for SHAs (when they still existed). One of the problems the team spotted was that complex models with very large amounts of detail tended to be very hard to audit properly and often contained errors in their code. That's bad when you rely on their outputs. 

But that complexity also had a side effect that is, though not an error, worse: they fossilised current assumptions about the mix of the workforce. In particular, they made assumptions about the need for very small specialist subgroups of staff (humorously like the number of orthopaedic surgeons specialising in only left hands). The problem is that practices often change faster than the model. So, if the model spits out the demand for some small highly specialised group in a decade's time, it may have been overtaken by major changes in the way that specialty works. Once upon a time, for example, most cataract operations were done under general anaesthetic. Then it became obvious that local anaesthesia was faster and safer and the mix of activity changed rapidly. Any model built before that change was obvious would forecast a completely incorrect mix of staff or number of staff.

When you build complicated models there is always a big risk that the assumptions in the model persist long after the change as many models are even harder to update than clinical practice. Or the modellers just don't notice the changes and the NHS keeps relying on their model as the users of the model don't understand it well enough to understand the assumptions it makes.

In another case I studied a model built for NICE on staffing in A&E departments (see my commentary). The original report gained a lot of credibility when NHSE allegedly suppressed it. But it was leaked alongside the full documentation on a simulation model built by external consultants that had been a major evidence source for their recommendations. I read the documentation. I wept. The assumptions about how an A&E worked had almost no relationship to reality and ignored very clear, well-known, data about actual performance. It looked like it had been built by someone who had never visited a real A&E or mapped a real world operational process. I suspect that most people who read the report didn't understand the model or that it was a major part of the evidence behind the recommendations. But bad models make bad recommendations. Worse, complex models make those mistakes harder to spot.

Even when a model works it may fail to influence the right decisions

The debate about the need for an NHS workforce model seems to assume that models have a magical ability to change the decisions people make. Decision scientists know this isn't true. 

Many analyses and models completely fail to influence actual decisions even when they are reliable and the data behind them is correct. For example, the NHS has reported on the hospital maintenance backlog for years, including an estimate of the need for urgent action to limit the immediate risk to patients. Yet decision makers have repeatedly chosen to spend far too little on capital (the budget has been about half that of peer health systems for most of the last two decades). And the high risk maintenance budget backlog grows every year. Maybe bad decision making is very resistant to modelling or analytical data.

Models work best not when they give a highly specific and precise answer but when they help decision makers to understand the core issues behind the decisions they have to make. A complex and detailed model of the NHS front line workforce is unlikely to achieve this. Not least because, if its focus is just the front line, it will fail to help decision makers to understand the tradeoffs involved in between different decisions they could make today.

What if, for example, a small increment in the number of managers greatly improved the productivity and quality of the work done in hospitals? How would that choice interact with the future needs for front line staff? We already know that more managers do have significant effects (see this summary from the NHS Confederation) but the idea that a workforce model should think about them is entirely absent from the current discussion on workforce modelling.

Also missing in the discussion on workforce modelling is any hint of how capital spending on better buildings, equipment or IT could contribute to productivity. But decision makers have to make tradeoffs today about how to split the budget among front line staff, support staff (including managers) and capital spending. And for most of the last decade that choice has skewed towards the front line leaving the NHS with a chronic deficit of support staff, managers and adequate modern buildings and IT (for some data see my analysis here). Those choices have led to declining front line productivity, a much worse working environment and, arguably, contributed to recruitment problems and higher staff turnover. A model focussed just on the long term needs for front line workforce numbers will encourage continued neglect of those other factors which directly impacts the immediate workforce.

Conclusion: a long term workforce model is a distraction not a solution

It seems obvious that the NHS has a serious shortage of front line staff. But that observation is very deceptive. It is a symptom of widespread problems of productivity and blocked flow of patients. As with many medical conditions, there is a strong temptation to treat the symptom and assume that this cures the disease. In NHS language, focussing on the front line workforce assumes that investment in the front line workforce cures the problem. But, like trying to cure headaches caused by a brain tumour with stronger painkillers, treating the symptom doesn't solve the underlying problem.

A model that assumes that frontline overload is cured purely by adding more staff distracts attention from all the other factors causing overload of front line staff. So attention will be distracted from inadequate buildings, obsolete equipment, slow and badly designed IT, admin overload caused by a lack of support staff and managers, and poorly designed clinical pathways. And "more staff" doesn't fix problems where the issue is having the wrong mix of staff. 

Some of those problems might be fixed, in principle. We could, perhaps, build a model that takes into account the staff mix, not just the overall number of staff. It could even highlight the areas where extra staff would most improve overall productivity (eg, to fix A&E overload, invest in staff who can improve the flow through beds). Unfortunately the first step would be to develop an analysis of how the whole system fits together and therefore identify which incremental investments would most improve the productivity of the system. There is no such analysis, though there are plenty of hints that workforce isn't the biggest problem. 

The workforce model as currently discussed seems likely to further distract the NHS from other major problems. A focus on the front line risks distracting from even bigger staff shortages behind the front line. And from a long term neglect of capital spending (leading to major issues with buildings, equipment, and IT). Furthermore there is a big risk that a long term workforce model could ignore the short term decisions that might help the immediate problems with workforce and could encourage the NHS to build in false assumptions that fossilise bad current choices.

That's a lot of risks for an unclear outcome. Whatever the intuitive attraction of a transparent model of workforce needs, it is far from obvious that the NHS would get anything useful.

We need a more informed debate about what holds back NHS productivity, not a model focussed on the front line workforce.

The biggest problem holding back NHS performance is a lack of investment in capital, innovation and management not staff or beds

In debates about what the NHS should do to improve its performance it is common to see benchmarks of staffing or bed numbers versus comparable health systems. The argument is that the system needs more doctors/nurses/beds. This is not the biggest problem. The NHS has suffered far more from a failure to invest in capital, innovation or management. 

Charts like the one below dominate the debate about NHS funding:

The message is simple: the NHS is underfunded and needs to have more doctors (or nurses or more money, depending on which lobbyist is producing the charts).

The implication is that the very obvious performance problems of the system would be solved if only we matched the same level of doctors/nurses/funding as the peer group of other health systems.

There is some debate on the strict comparability of these metrics and that is rarely mentioned by the lobbyists. In fact I once saw Nigel Edwards heckled for pointing out that a major revision on OECD metrics of overall spend on the NHS and comparable systems made the NHS look middling rather than an outlier on the low side. People really want the explanation for poor NHS performance to be simple and, preferably, a conspiracy they can blame on the Conservatives. It is frequent to see phrases like "Tory staffing cuts".  And that is the end of the debate for many.

But, even though the overall NHS budget saw unusually low growth in the decade after 2010, front line staffing increased significantly. The "tory cuts" in staffing led to ~30% more doctors and 15% more nurses:

I don't want to argue that the government is not to blame. I do want to argue that we are looking in completely the wrong place about where policy has been wrong.

There are far, far bigger problems than not having enough front line staff. And the real problems that front line staff experience are often consequences of those other failures. Worse, those other failures get almost no attention and the topics that do get attention distort the debate so much that the biggest and most important failures get essentially no critical attention.

Before I get to the other issues that I argue matter more than staffing, it is worth a quick review of why the staffing benchmarks are not that relevant to NHS performance.

One reason is a lack of strict comparability with the other countries in how they use staff. But a more interesting one is that the implied policy when it looks like the NHS is behind ("fix the staffing, fix the problem") very clearly doesn't work in specific cases. An example is A&E staffing. It has been repeatedly claimed in the last decade that the decline in A&E performance is a consequence of a lack of A&E staff. But the number of A&E doctors rose faster than demand every year since 2010 and the number now is >30% higher than it was in 2010. The number of A&E specialists grew faster than any other for most of the last decade. But performance has declined monotonically over that period. 

In this case it is easy to see why staffing was never the problem. Patients suffer long waits in A&E not because there are not enough A&E staff to treat them but because there are no free beds to move the patients to. This was known in the early 2000s when the 4hr target was set and has repeatedly been shown in detailed analysis of the causes of waits in the last decade (see, for example, this from Monitor). To put it simply, more A&E doctors can't magic up more free beds. They mostly can't even influence the number of free beds. So adding more staff doesn't fix the underlying problem. (I should add that the longer the A&E queue, the more staff are needed to handle it. So the apparent staff shortages are actually a symptom of a different problem elsewhere in the hospital, not the cause of the problem. Adding more doesn't fix the problem.)

So what other errors has the NHS made?

The NHS benchmarks that really matter

Let me tell you the answer before working through the details and arguments. 

The NHS might be slightly short of front line staff but it is catastrophically short of support staff like managers and has a long standing lack of investment in innovation and capital spending. If benchmarks versus comparable health systems are any guide, the NHS is a spectacular outlier in investment and management. 

The NHS is very undermanaged

When the Lansley Reforms were in draft form and proposed severe cuts in management numbers there were several analyses that criticized this goal. The King's Fund concluded as part of its commission on leadership and management that:

"If anything, our analysis seems to suggest that the NHS, particularly given the complexity of health care, is under–rather than over–managed."

And this was before the drastic cuts in the Lansley reforms were implemented (see the staffing chart above for their immediate impact). A more recent analysis (in a report by Ian Kirkpatrick and Becky Malby for the NHS Confederation) said:

"...it is hard to argue that the NHS is ‘overmanaged’. At approximately 2 per cent, managers are a very small proportion of the NHS workforce. By comparison, ‘managers, directors and senior officials’ in the UK as a whole make up 9.5 percent of the workforce." 

And, in another report in the series they argued this: 

"We found that even a small increase in the proportion of managers employed (from 2 to 3 per cent of the workforce in an average acute trust) had a marked impact.

Up to a certain point, larger management functions in trusts were associated with higher patient satisfaction scores, a 5 per cent rise in hospital efficiency and a 15 per cent reduction in infection rates. Further tests revealed that it was primarily higher levels of managers employed that drove these improvements and not the other way round."

Both NHS management numbers and total administration costs are far below international norms and have fallen sharply since 2010. Given the clear evidence from Kirkpatrick's work that more managers make hospitals function more effectively, cutting management looks like a bad error. Adding more managers has a lot of leverage over the performance of all the other staff but the NHS has chosen to cut their numbers instead.

Both manager numbers and total administrative costs are far more out of line with international norms than the number of doctors, nurses or beds. Although good comparable metrics are hard to find, the IFS claimed this in 2018:

The OECD has compiled data on administrative costs of different health care systems at the ‘macro’ level – which captures the amount spent on planning, funding and monitoring care, but not administrative costs within individual hospitals. 

They found that the NHS spends relatively little on overseeing and planning care, relative to other comparable systems. In 2014, the UK, Portugal and Ireland all devoted 1.5% or less of their government or compulsory health care expenditure to administration. This compares with an average of 3.1%, with 4.1% in France, and 7.9% in the United States.

Investment in innovation and capital is grotesquely low 

Another area where the NHS is an outlier is spending on capital and innovation. In fact it is an extreme outlier.

This is somewhat ironic given Rishi Sunak's recent spring statement where he diagnosed a major problem for the UK economic productivity as being largely caused by a lack of investment in innovation and capital. As he argued:

"Over the last fifty years, innovation drove around half the UK’s productivity growth.

…our lower rate of innovation explains almost all our productivity gap with the United States.

Right now, we know that the amount businesses spend on R&D as a percentage of GDP is less than half the OECD average.

…

Weak private sector investment is a longstanding cause of our productivity gap internationally:

Capital investment by UK businesses is considerably lower than the OECD average of 14%.

And it accounts for fully half our productivity gap with France and Germany."

The NHS is the largest part of the UK economy controlled by the government. So exactly how has the government sought to control its spending on the key factors that determine productivity in the rest of the economy? 

It won't surprise you to know that it is the opposite of the Sunak recipe for productivity in the private sector.

The Health Foundation did a (little read) briefing on this in 2019. It starts by pointing out the obvious:

"Capital spending is a critical input in health care, with new technology able to transform services and improve workforce productivity. 

The DHSC has proposed a more technology–and data–driven NHS. New technology and IT could improve patient services and increase productivity, but both currently make up a small proportion of capital spending."

So the DHSC has an ambition to exploit technology and IT (which needs investment). But the Health Foundation analysis of 20 years of NHS capital spending compared to peer health systems looks like this:

Only during the late Blair/Brown years did NHS capital spending come close to international norms and it is often the lowest or next to lowest in the whole dataset. The Health Foundation  argue:

"For the UK to move up to the average for OECD countries, capital spending would have to almost double as a share of total health spending"

The National Audit Office also reviewed capital spending in the NHS in 2020 and some of their analysis tells an even more sorry tale. 

They start by pointing out part of the current situation with buildings and other capital assets:

Parts of the NHS estate do not meet the demands of a modern health service. NHS hospitals include Victorian-era buildings, and 14% of the NHS estate predates the formation of the NHS (1948). 

…

The growth in backlog maintenance indicates that there is an increased risk of harm to patients … the backlog of maintenance work to restore buildings to an appropriate standard was around £6.5 billion … High-risk backlog maintenance currently stands at £1.1 billion, and grew by 139% between 2014-15 and 2018-19, indicating an increased risk of harm to patients.

It isn't just the lack of modern IT and diagnostic equipment that holds back the NHS. It needs more modern buildings to do a good job but frequently doesn't have them.  Worse, in some cases, maintenance problems in the existing buildings are so bad they risk immediate harm to patients.

Given this already disturbing background it might be a surprise when they point out that in many years of the last decade the already inadequate capital allocation to the NHS was underspent:

"Between 2010-11 and 2012-13, there was an average underspend of £677 million (12%) against the capital spending limit. In 2017-18, £360 million (6%) was unspent."

And their story gets worse:

"Since 2014-15 the Department has transferred £4.3 billion from capital to revenue spending"

So not only does the NHS start with an inadequate budget, which it underspends, it is then encouraged to pilfer the capital budget to cover operating costs. The reason this has been encouraged is because it is a convenient short term way to cover up operating deficits. These are embarrassing. And, obviously, avoiding embarrassment is more important than the roof of an operating theater falling in. Better still, while operating deficits are visible every year, most of the catastrophes from the maintenance backlog will appear slowly over a decade.

It might seem strange that the inadequate capital budget should ever be underspent. But the NAO explain that too by pointing out that the paperwork and bureaucracy of applying for capital is so baroque that many hospitals can't even get their cases for urgent maintenance past the system. And, even if they do, they might lose the allocation if they can't spend the money in-year as next year's allocation may be arbitrarily different. Short term changes to the budget every year make long term planning of capital spending impossible.

The buildings are inadequate for modern healthcare activities. There is a huge and rising backlog of maintenance to keep the show on the road that is limiting the capacity of the system to do more work. The system frequently steals from capital to employ more staff who will have to work in an environment where their work will be harder and less productive. The NHS is close to bottom on international rankings of the amount of high-tech equipment it needs to do the diagnostics necessary to tackle long elective queues. And there is little budget for investing in better IT to enable front line staff to work faster and more productively without the burden of coping with decade-old kit.

It is lucky that investing in capital or innovation doesn't matter for productivity. Oh, wait, that's exactly what the chancellor blamed for low productivity.

As an ironic coda to this section the following story appeared in the HSJ the day after his spring statement:

"Tech spend under pressure as NHSE told to ‘cut core funding’"

It hasn't taken long for the promises in the last spending review to invest more capital in the NHS to hit the buffers of old Treasury munchkin habits.

The government's goals for an improved, more productive NHS are directly undermined by its choices about how to allocate resources.

There are several important messages here.

Problem 1: the government itself has correctly argued that spending on capital and innovation are vital for driving up productivity. So much so that they are increasing the incentives to encourage more such spending in the private sector. But, when they control the budget, they do exactly the opposite. And, at the same time, continue to demand even higher productivity gains from the NHS. Whipping a dead horse doesn't cover it. It is more like whipping the reliquary containing the ashes of the horse cremated a decade ago after being euthanized for breaking a leg.

The second huge problem here is the lack of attention this analysis has had among commentators and the media. The news is full of stories about how the NHS is struggling because it has 10 or 20% fewer doctors than comparable systems. Lobbyists for nurses and doctors demand higher levels of staffing to fix the overwork, the current catastrophic waiting lists and A&E delays. But there is little mention of the fact that the NHS has perhaps half the capital employed per worker than almost any other health system. The NHS is a far more extreme outlier on this than it is on staffing. And the day to day work the front line staff have to do is much harder and less productive as a result.

The third problem is management. NHS management does make the news more often than capital spending, but almost always to disparage it. Stories often argue that we could cut management even further to put more resources to the front line. These stories usually fail to note that NHS management has already been sharply cut by the Lansley reforms and is currently another major outlier in comparisons between the NHS and other systems (I made the case about managers and their importance in the NHS here). Nor do they mention that very clear work shows that more managers make productivity and medical quality better. 

In short, there is a fundamental mismatch between the government goal for a more productive NHS and the way it provides the tools to the NHS to achieve that productivity. The government and NHSE are like an army who recognises the need for bullets but forgot to allocate a budget for the guns required to fire them. 

The perpetual failure to invest enough in capital, innovation or management is a far bigger problem for the NHS than any shortage of staff. It is about time the commentariat, the media and the government realized this. We won't fix the NHS until they do.

 

How busy are English GPs?

Our GPs are probably grossly overworked. There is plenty of evidence that this is true, but recent data collected by NHS Digital paints an ambiguous picture. Many GPs have reacted to my analysis of that data with incredulity, claiming NHS Digital don't know what they are doing. Or that the data is meaningless and too much of a burden to collect. The real situation is more complex and won't be fixed by NHS Digital alone but requires GPs to pay more attention to how they collect data and why it is collected. And it is critically important that the system does a better job of collecting data about their activity or the case for higher primary care funding will fail as soon as it is examined by the treasury. GPs, their system providers and NHS Digital all need to work together to create more useful data.

Recently released data poses some interesting questions for GPs

At the start of 2019, NHS Digital released the first public version of a new dataset on GP activity they had been collecting since the end of 2017 (and they are currently updating monthly). This dataset summarizes the number of consultations each day using data collected directly from the (major) providers of GP clinical systems (not all of the minor providers are covered yet). 

The public version of the dataset covers the CCG-level aggregate number of appointments and the status of those appointments in several different categories:

• The appointment mode (face to face, telephone, visit, video etc.)
• The appointment status (basically whether the patient turned up for a booked slot or not)
• The type of staff who saw the patient (essentially whether the person who saw the patient was a GP or someone else)
• The delay between the booking and when the appointment actually happened

The data coverage is good for most CCGs (only one has no data at all) and there are some big, interesting insights in the data. For example, the number of no-shows ("did not attend" or DNAs) is very strongly related to the length of time a patient has to wait for an appointment (see my analysis here and a discussion in the BMJ here including comments on why the results are different to some previous academic analyses of DNA causes).

But the analysis I did on the number of appointments per day done by GPs caused a much bigger kerfuffle. I want to be fair and present those criticisms alongside the analysis. It is possible that both the analysis and GPs criticisms of it are right, but, if that is correct, then GPs, their systems providers and NHS Digital need to make some significant changes to how they work.

Before we start, a thought experiment

To put the results in context, it is worth doing a simple thought experiment to put the results in perspective.

Imagine a GP who does nothing other than see patients. She fills her entire 8-hour working day with 10-minute appointments with no breaks. Let's also assume that she works 48 weeks out of 52 (including bank holidays in the 4 weeks of downtime). And let's assume that each day worked consists of 48 appointments (this is how many fit into an 8-hour day if she takes no breaks between appointments or for lunch and does any other work after hours or at weekends). That means she does 11,520 appointments per year or about 44 per weekday in the year (she doesn't work every weekday because of holidays).

That 44 appointments per day doesn't sound like a sustainable workload. So let's assume a 1hr lunch break and gaps between appointments that brings the average work done down to 5 appointments per hour. Now she does 8,400 appointments per year or about 32 per weekday averaged over the year. That's probably still not sustainable every year but it is a thought experiment and we can make the simplifying assumption that GPs are superhuman. Bear this number in mind when we look at the actual reported workload from the NHS Digital data.

And, don't forget, that a real GP also has a ton of paperwork to process plus they need time for professional development, time to train new GPs and other staff and time to manage the practice. So lots of other work to fit into the actual working day alongside the 8-hr shift spent actually doing appointments. And, remember this analysis simply takes the total number of appointments recorded in a CCG divided by the number of weekdays and the number of FTE GPs; it ignores all the other work and all the extra time GPs put in beyond normal hours or at weekends.

What does the NHS Digital data show?

The original claim that prompted me to look more closely at the NHS Digital data was a simple calculation about the number of appointments per GP based on the headline numbers. NHS Digital estimate that about 307m appointments were recorded in the year to April (they adjust the total for missing data and their adjustment seems to differ from mine by about 10m/year but this is not a huge gap given other uncertainties in the data). Taking their number per working day in the year and adding the March estimate that there are 33,425 FTE GPs in England suggests that the average GP does about 35 appointments per working day. This seems to confirm, when compared to my thought experiment above, that the typical workload is well into unsustainable territory with a side order of "if we keep this up we are all going to die of overwork".

But that isn't what the dataset actually says. GPs have a lot of other staff who handle >40% of the patient contacts according to the data (this is the national average but see the map below for the variation among different CCGs in the % seen by non-GPs).

If you take that into account, the data actually says the average GP does something closer to 20 appointments per day. 

20 appointments/day is the England average. But, usefully, we have the staff census data that tells us how many FTE GPs there are in each CCG and the appointment counts are available for each CCG as well. So I can do a CCG-level analysis and look at the variation across the country (taking into account the number of GPs in the CCG and the % of appointments with a GP). If we generously assume that activity where the staff type for the staff fulfilling the appointment is "unknown" is done by GPs we get the following distribution:

(Note: each block on this chart is a single CCG and the activity data is based on activity done on weekdays in the first 4 months of 2019. Also note that an interactive version of the dataset and many analyses is available here and you can test how varying some of my assumptions changes the results.)

In about 8 CCGs the GPs appear to be seeing an average of about 26-30 patients per day. The commonest CCG average is, however, closer to 20/day. Also, don't forget, the results are averaged over at least a month and reflect the average activity on all available days, not just the busy days where all the GP time is devoted to seeing patients.

Don't forget, though, that this is the average across a whole CCG: the variation among practices within each CCG is probably large so there will probably be some practices where the GPs do >30/day even when the CCG average is just 20.

Some GPs were indignant when I put an early version of this chart on Twitter. The typical response was "20 a day! I do far more work than that in a typical day. NHS Digital don't know what they are doing. The data is obviously rubbish…". I'm not entirely convinced that all the criticisms are fair, but it is worth looking at some of them to see if there are lessons to be learned.

Possible problems with the data

This just doesn't reflect my typical day

This could be because the data is just wrong, but we will come to that possibility later.

Another possible explanation is that what GPs remember about how busy they are isn't an accurate reflection of the average number of appointments across the whole month or year. If, for example, what GPs remember are the day when they are focussed on seeing patients but they also devote other days to practice management, paperwork, training or some other work that doesn't involve booked appointments, then their memory won't match the averages reported here: the results will reflect the total appointments divided by the total available time for all work not just the typical day when the GP is dedicated to seeing patients. 

This work (and my initial thought experiment) assumes a 40hr week. Many GPs spend a lot more than 8hrs a day working (perhaps doing all the admin in their unpaid overtime). This dataset is not recording that work or adjusting the available hours to account for it. This doesn't impact the results but it may make the perception of the results much less clear than they should be.

The data clearly is missing a lot of the work we do

Part of this is a perception problem as discussed above. But it could clearly be true that a lot of activity is not correctly recorded. 

But the blame here isn't on NHS Digital. The data is directly collected from the major clinical systems used in practices to record activity. If the numbers are a big understatement of the appointments being booked, then practices have a big problem not just NHS Digital. I suspect we are not understating the true number of appointments by a large margin.

But we could be understanding the amount of activity. Many patient contacts don't result in a booked face-to-face appointment. GPs do some work online or by telephone. Clinical systems are not uniformly good at recording that activity as I know from comparing incoming requests via askmyGP (an online tool that manages all incoming patient requests and helps GPs to completely change their workflow when responding to those requests in ways that improve the speed of responses to patients and lowers the degree of overwork for GPs). Successful online tools (and some phone-triage approaches) often result in only 1 in 3 requests leading to a face-to-face appointment. This reduces the number of appointments recorded in the clinical system while providing satisfactory responses to the other patients. But much of the activity for those other responses may never appear in the clinical systems. As GPs adopt new ways of working their clinical systems may become a lot less good at measuring their activity.

On the other hand, the NHS Digital data does record a lot of phone activity. Between 10 and 15% of all appointments are recorded as taking place online, by phone or by video. We know that recording of this is patchy, but so is uptake of alternative ways of responding to patient demand. Some GPs still think that face-to-face appointments are the only response despite mounting evidence that offering a wider range of alternatives can reduce GP workload and make patients happier because they get a faster response.

We have to admit, though, that traditional GP systems have not caught up with this change in practice and won't consistently record the shift in activity reliably.

NHS Digital will draw all the wrong conclusions from these results

I got the impression from some comments on my initial analysis that, not too exaggerate too much, central NHS bodies are just a bunch of malingering bureaucrats who exist just to malign hard working GPs by misrepresenting how hard they work by providing the NHS leadership with false, irrelevant data about what they do. And that any suggestion that this should be fixed by GPs spending more time to provide accurate data about what they do would be yet another straw added to the already broken camel's back that is the typical GPs unsustainable workload (or some such mixed metaphor for even more overwork).

But NHS Digital's motivation is not to burden GPs with more work or to undermine the case for more GPs. It is to understand what GPs are doing to develop better policy, and that can't be done without better data. Originally the idea was focussed on developing a better understanding of winter pressures but it should be useful for bolstering the case for more investment in primary care or for promoting better working processes. 

But, imagine the conversation the NHS would have with the treasury when bidding for the money to recruit another 5,000 GPs:

NHS "We need another 5,000 GPs."

HMRC "How much will that cost?"

NHS "A couple of billion pounds a year."

HMRC "That sounds like a lot. Why do you need it?"

NHS "They keep telling us they are overworked."

HMRC "That's what everyone says. How much work do they do? Show me some numbers."

NHS "They say they are doing unsustainable numbers of appointments a day."

HMRC "How many is that."

NHS "We don't know exactly but more than 30/day doesn't sound sustainable"

HMRC "So how many are they actually doing?"

NHS "No idea. But I'm sure it is a lot. We really need the money to recruit more GPs."

HMRC "Sure. You can have it when we cash the £350m/week we will save from Brexit."

NHS "Really?"

HMRC "No." 

HMRC "We were joking. Sod off and don't come back until you have some actual evidence."

OK, I'm parodying the case for more money for GPs and there are other hard sources of evidence that say they are overworked. But if the NHS as a whole doesn't understand what, in detail, GPs are actually doing any case for reform or extra cash is going to look weak.

There are more reasons than this to have reliable data. GPs need it to understand their own work or they won't have any idea how to do a better job. While the NHS Digital appointment data doesn't capture everything they do, it is at least a start providing a better understanding of a large part of their activity. And the main limitations that prevent the data being more useful are not NHS Digital's fault but the fault of the system providers and choices made by GP practices. 

The data isn't consistent or comprehensive

GP Clinical systems are built around a model of GP work that assumes everything revolves around a pre-booked face-to-face appointment. GPs can vary how they label the slots for activity in the system and they don't always assume every appointment is 10mins long (though their royal college doesn't seem to have caught up with this realisation). Systems usually allow the exact time taken for each appointment to be recorded (assuming GPs press the right buttons which they don't always do). Some systems make it easy to record activity in open-access clinics. But systems tend to assume every appointment slot is the same length and some are completely, irretrievably hopeless at measuring the time taken for consultations done by phone. 

One of the biggest problems faced by NHS Digital is the number of different descriptions given as labels for each slot. Practices are, essentially, free to label each slot however they want and one estimate says that there are tens of thousands of different labels in use across practices in England. So when NHS Digital try to acquire data from everyone it is a gargantuan task to correctly identify the different types of appointment and group them together in ways that make sense. Was that a face to face pre-booked appointment? Or an ad-hoc on the day f-to-f as part of a walk-in clinic. Or a slot where a nurse did flu-jabs?

And, as more activity is initiated online and GPs choose to triage patient requests before booking appointments, there are more varied ways for GPs to respond to patients. And the systems–designed as they are around the idea that most activity involves pre-booked slots–don't record all those interactions. In practices using online tools like askmyGP only around ⅓ of requests result in a face-to-face appointment (e-Consult, another online tool provider estimates a similar ratio). This means, potentially, that the majority of GP activity won't be recorded in the traditional clinical systems.

So it may well be true that what NHS Digital are collecting is not a reliable guide to what activity GPs are doing. But the fault can't be fixed by them: if we want better data the problem has to be addressed by the practices (who need more consistency is how they use their systems) and the suppliers (who need to design their systems to break the model that everything neatly fits into 10 or 15min slots).

Some suggestions about how to do a better job

It is not at all helpful to complain that the NHS Digital GP data is rubbish. If we want to do a better job in primary care (to improve how GPs work and make an unanswerable case for more investment) we need better data. 

An article in the Economist describes some improvements in primary care and how they were done:

When the four practices serving St Austell merged in 2015, it was an opportunity to reconsider how they did things. The GPs kept a diary, noting precisely what they got up to during the day. It turned out that lots could be done by others: administrators could take care of some communication with hospitals, physios could see people with bad backs and psychiatric nurses those with anxiety. So now they do. Only patients with the most complicated or urgent problems make it to a doctor.

The key point here is that only when they had reliable data about their activity could the GPs redesign the work they did in ways that would improve it.

As a recent Nuffield Trust comment on the GP dataset says:

In hospitals, routine datasets such as Hospital Episode Statistics (HES) give us insight into who gets treated, for what, by whom, and where. This basic knowledge of the activities and performance of hospitals is now essential for making evidence-based policy changes, monitoring existing policies and providing fundamental administrative data to run the system and allocate funds. There is a clear need for something similar in general practice…

But, they continue:

NHS Digital makes it clear that what has been released so far still needs improvement. As it stands, there are a number of problems with the data and these limit the extent to which we can use it as a resource for evidence-based policy. 

Having different practice IT systems gives GPs flexibility over the way they organise their work. But the absence of reporting standards in general practice means information is recorded differently in each system, making it difficult to combine the data, and information gets lost.

In the early 2000s, hospital activity data (HES) was in a similar state of disorder. Doctors took little interest in making sure that the amount of activity recorded was correct or the clinical details correctly coded. They too argued that this was just an unnecessary bureaucratic task required puerly for "feeding the machine" (even though the most basic metric of hospital success–not killing your patients too often–requires accurate coding). 

But HES got a lot better. One way it got better was to show medics what HES said about the work they did. The Royal College of Physicians ran some interesting experiments starting in 2002 (reported here and here). They took HES data and fed it back it to the clinicians whose activity it recorded. In one of the reports on this exercise they argue this:

A vicious circle ensues: routinely collected data is perceived as being of poor quality and unable to support the needs of the individual. Individual clinicians avoid the use of such readily available information... Centrally held datasets remain unchanged through neglect, clinicians failing to engage with the information process in their trusts and remaining ill at ease with the records of activity which result. It is clear that if this cycle is to be broken, steps must be taken to engage clinicians at a level whereby the information is made readily available, accessible in format and of use to clinical practice. By examining routine data from a clinical perspective and feeding issues of quality back to trust information departments the cycle can be reversed.

Hospital clinicians being shown their own data in a digestible form was a big part of the drive that improved the quality and utility of HES data. GPs and NHS Digital need to do something similar.

Perhaps NHS Digital could take the first step by making practice level data from the central collection available to practices in a useful, comprehensible format so GPs can see what the data actually thinks they did. This would be the first step in driving improvement in what is recorded.

Conclusion

In short, here are some simple ways for the different players in primary care to change what they do in ways that would promote real improvement for GPs and their patients.

NHS Digital should make a friendly, easy to comprehend, version of their dataset available to every individual practice so GPs can compare what they think they did to what NHS Digital's extract has recorded.

GP System Providers should make their systems more flexible in recording non-appointment activity and should work to support GPs to use more consistency in how they record and label their activity.

GPs and practices should not just dismiss the data as being unreliable and useless. They should strive to understand it and seek ways to work with NHS Digital to make the data useful for local purposes and more reliable for policy making.

Everyone should not dismiss data they don't like but engage with it and seek to make it both more reliable and more useful.