We are having the wrong debate about modelling the NHS workforce

It sounds intuitive that having a good model of the NHS workforce would be useful for solving many observable problems in the current NHS. But there are reasons to assume any such model would not be as useful as expected and could even be harmful. More importantly, the starting point suggested for the model is wrong in multiple ways that almost certainly guarantees the model would fail to solve the real problems.

When I first heard that a coalition of Labour MPs and the ex-NHS SoS Jeremy Hunt were proposing an amendment to the Health Bill to insist on regular publication of a workforce model for the NHS, I thought the idea was a good one. Better, transparent information about the staffing needs of the future NHS sound like a useful idea to test against government policy. 

But, when I reflected on some of the issues I have seen in the past when Strategic Health Authorities did workforce plans I started to have doubts. Then, after some further conversations and cogitation, those doubts grew. Considered alongside my analysis of what the biggest challenges are for the NHS (in short, the front line workforces is far from the biggest problem), my skepticism strengthened. 

So, I'm going to argue that, while a good workforce plan might help, the one we are likely to get is likely to be somewhere between useless and harmful. It is starting in the wrong place, has unclear goals that will likely make it far less useful than expected and has some risk of making things worse. 

That's a big claim. Let me walk through the potential issues I see step by step. 

Where the workforce model might go wrong

It doesn't start by considering productivity

The starting assumption in the debate is the almost universal belief that the problem is a shortage of front line staff. Commentators observe busy A&Es, overwhelmed GPs risking staff burnout, hospitals where waiting lists are growing not falling, and leap to the conclusion that the only way to address these is more medical staff.

But the problem framed this way distracts from any analysis that concludes anything other than "more staff" can influence the amount of work done. Concluding that only more staff matters rules out many known interventions that should be part of the debate, especially those that improve productivity..

Here is a simple example. NHS A&Es are currently catastrophically crowded and patients are getting treatment so slow it is killing them. But we know from analysis that has existed since the 4hr target was introduced that the number of A&E doctors has very little influence on the speed (for a fun review of the evidence on this and how little appetite the system has to listen to it read this BMJ piece and the replies). The dominant cause of long waits in the last decade has been slow access to free beds for admitted patients. The problem isn't even inside the A&E, so adding more A&E staff won't fix it. To be fair, the workload needs in A&E do increase sharply with the length of the queue. But they don't help to make the queue shorter. So any workforce model that ignores the external factors causing the queue will end up recommending far more A&E staff than needed if the external bottleneck causing the queue is ever solved.

Another example shows that not thinking more widely about the mix of staff required is a major problem in planning. A Royal College of Surgeons blog reported in 2017 that the productivity of surgeons had declined sharply as their numbers rose because numbers of support staff and nurses had not risen. It reports "Between 2010 and 2016, consultant numbers rose by 22%, compared to just 1% for nurses and 2% for all staff."  It Also pointed out that "... consultants in hospitals that invested more in infrastructure and building … are more productive". (it is worth reading the original analysis by the Health Foundation as well for much more detail). So, is a workforce model is built to forecast the number of surgeons required, but ignores the number of nurses and support staff or the capital required to create a productive theatre, it will vastly overestimate the number needed.

The point is that productivity depends on the mix of staff and other factors like equipment. Driving higher output across the NHS requires a good understanding of where the bottlenecks to productivity are so the right mix of capital and people can be deployed. Adding more of the most visible front line staff is often not that effective. But it is what a "more resources to the front line" workforce model is likely to achieve.

It is unclear what decisions a long term workforce model is intended to support

The only point of any model is to support better decisions. If you are vague about what decisions, then the model is likely to be unhelpful and even misleading. 

So what decisions could a long term workforce model support? So far the discussion has tended to focus on the need for a model to inform the NHS about its workforce need in 5, 10 or 15 years. 

What decisions could such a model influence? Not many. If we know we need more nurses in 5 years time the NHS might just have enough time to increase the number of training places to increase the numbers qualifying in 5 years time. But it is unclear whether increasing the number of doctors in training right now would lead to higher available numbers in a decade's time. 

If the NHS is short of particular skills right now, it is unclear how a long term model can possibly help. What the system needs most urgently is some idea of what the options are today. 

If the model focuses–as much of the discussion about it has–on front line staffing need, then it also misses critical groups that contribute to the productivity of the front line (see the section on productivity for why this is important). The question that desperately needs an answer is what different mix of staff, equipment, buildings and new clinical processes would give the biggest increase in the number and quality of treatments delivered. Many of those questions are not workforce questions at all and even a workforce-only model needs a good understanding of how different staff interact to make the front line more productive. That understanding will only come from a significant piece of careful analysis that doesn't seem to exist. 

What the NHS needs right now is that analysis. It needs to know where the bottlenecks to higher activity are. It needs to know what mix of capital spending, front line staff, support staff and managers would lead to the largest improvement. Without that a workforce plan will be about as useful as a one-legged trapeze artist with an itchy bum.

A workforce model designed to tell the system how many staff it needs to put into training now will get the wrong answer because it ignores all the interactions and other factors that matter and, even if that wasn't true, could not influence any decision that will have an effect for 5-10 years at best.

Major factors that influence the workforce today are likely not part of the model

The NHS has a workforce problem right now. And many of the factors causing problems are not relevant to the long term supply of qualified staff. Or anything else likely to appear in the proposed workforce model.

Right now the biggest factors influencing staffing gaps are recruitment problems and high turnover (plus illness, if temporary pandemic-specific problems count). These problems don't just affect the front line staff groups but are common in the other groups where a lack of staff has a lot of leverage over front line productivity.

There are many causes of recruitment problems and high turnover. In some groups NHS pay is inadequate compared to other jobs the same staff can do. This is a big issue for nurses but a huge problem for support staff like data scientists. It is also a bigger problem in some geographies like London where the cost of living is much higher and there are more alternative well-paid jobs. But the NHS finds it almost impossible to flex salaries to retain the people it needs both because the pay scales are national and because some groups are vastly undervalued in AfC grading compared to the market.

Working conditions are also a huge factor for recruitment and turnover. If the space is badly adapted to the work being done (~14% of buildings predate the NHS!) then the environment will be poor. Badly maintained buildings add to this (the maintenance backlog is about £10bn). Old, shonky equipment is slower and harder to use than modern equipment. IT systems are often slow and not seamlessly integrated so staff waste time waiting to log on or logging in to a dozen separate systems to complete a clinical task. Front line staff end up spending too much time doing tasks that should be done by support staff or managers (where staffing levels have been cut to "put more staff on the front line") instead of caring for patients. 

Too much of the people management in the NHS is bad. Staff are treated badly and insensitively by managers but also by senior doctors and nurses (the Ockenden report didn't just blame "staff shortages", it clearly blamed senior staff of all professions for ignoring clear signals about problems and even suppressing whistleblowers). 

Very few, if any, of the factors that discourage recruitment and drive high turnover are part of any proposed workforce model.

So the model won't tell the NHS whether a big increase in capital spending, creating better buildings, equipment and IT systems, would yield rapid gains in a better working environment. Nor will it conclude that recruiting more support staff to enable the front line to focus on treating, rather than admin paperwork, would improve their job satisfaction. And nobody in NHSE would allow the model to conclude that salary flexibility might yield immediate benefits in both lower turnover and higher recruitment rates.

That means that the model is likely to have nothing to say about the major factors that could impact the workforce any time in the next 5 years. What was the point of it again?

A long term workforce model risks fossilising current mistakes and practices

More than a decade ago I was part of a team auditing some workforce models for SHAs (when they still existed). One of the problems the team spotted was that complex models with very large amounts of detail tended to be very hard to audit properly and often contained errors in their code. That's bad when you rely on their outputs. 

But that complexity also had a side effect that is, though not an error, worse: they fossilised current assumptions about the mix of the workforce. In particular, they made assumptions about the need for very small specialist subgroups of staff (humorously like the number of orthopaedic surgeons specialising in only left hands). The problem is that practices often change faster than the model. So, if the model spits out the demand for some small highly specialised group in a decade's time, it may have been overtaken by major changes in the way that specialty works. Once upon a time, for example, most cataract operations were done under general anaesthetic. Then it became obvious that local anaesthesia was faster and safer and the mix of activity changed rapidly. Any model built before that change was obvious would forecast a completely incorrect mix of staff or number of staff.

When you build complicated models there is always a big risk that the assumptions in the model persist long after the change as many models are even harder to update than clinical practice. Or the modellers just don't notice the changes and the NHS keeps relying on their model as the users of the model don't understand it well enough to understand the assumptions it makes.

In another case I studied a model built for NICE on staffing in A&E departments (see my commentary). The original report gained a lot of credibility when NHSE allegedly suppressed it. But it was leaked alongside the full documentation on a simulation model built by external consultants that had been a major evidence source for their recommendations. I read the documentation. I wept. The assumptions about how an A&E worked had almost no relationship to reality and ignored very clear, well-known, data about actual performance. It looked like it had been built by someone who had never visited a real A&E or mapped a real world operational process. I suspect that most people who read the report didn't understand the model or that it was a major part of the evidence behind the recommendations. But bad models make bad recommendations. Worse, complex models make those mistakes harder to spot.

Even when a model works it may fail to influence the right decisions

The debate about the need for an NHS workforce model seems to assume that models have a magical ability to change the decisions people make. Decision scientists know this isn't true. 

Many analyses and models completely fail to influence actual decisions even when they are reliable and the data behind them is correct. For example, the NHS has reported on the hospital maintenance backlog for years, including an estimate of the need for urgent action to limit the immediate risk to patients. Yet decision makers have repeatedly chosen to spend far too little on capital (the budget has been about half that of peer health systems for most of the last two decades). And the high risk maintenance budget backlog grows every year. Maybe bad decision making is very resistant to modelling or analytical data.

Models work best not when they give a highly specific and precise answer but when they help decision makers to understand the core issues behind the decisions they have to make. A complex and detailed model of the NHS front line workforce is unlikely to achieve this. Not least because, if its focus is just the front line, it will fail to help decision makers to understand the tradeoffs involved in between different decisions they could make today.

What if, for example, a small increment in the number of managers greatly improved the productivity and quality of the work done in hospitals? How would that choice interact with the future needs for front line staff? We already know that more managers do have significant effects (see this summary from the NHS Confederation) but the idea that a workforce model should think about them is entirely absent from the current discussion on workforce modelling.

Also missing in the discussion on workforce modelling is any hint of how capital spending on better buildings, equipment or IT could contribute to productivity. But decision makers have to make tradeoffs today about how to split the budget among front line staff, support staff (including managers) and capital spending. And for most of the last decade that choice has skewed towards the front line leaving the NHS with a chronic deficit of support staff, managers and adequate modern buildings and IT (for some data see my analysis here). Those choices have led to declining front line productivity, a much worse working environment and, arguably, contributed to recruitment problems and higher staff turnover. A model focussed just on the long term needs for front line workforce numbers will encourage continued neglect of those other factors which directly impacts the immediate workforce.

Conclusion: a long term workforce model is a distraction not a solution

It seems obvious that the NHS has a serious shortage of front line staff. But that observation is very deceptive. It is a symptom of widespread problems of productivity and blocked flow of patients. As with many medical conditions, there is a strong temptation to treat the symptom and assume that this cures the disease. In NHS language, focussing on the front line workforce assumes that investment in the front line workforce cures the problem. But, like trying to cure headaches caused by a brain tumour with stronger painkillers, treating the symptom doesn't solve the underlying problem.

A model that assumes that frontline overload is cured purely by adding more staff distracts attention from all the other factors causing overload of front line staff. So attention will be distracted from inadequate buildings, obsolete equipment, slow and badly designed IT, admin overload caused by a lack of support staff and managers, and poorly designed clinical pathways. And "more staff" doesn't fix problems where the issue is having the wrong mix of staff. 

Some of those problems might be fixed, in principle. We could, perhaps, build a model that takes into account the staff mix, not just the overall number of staff. It could even highlight the areas where extra staff would most improve overall productivity (eg, to fix A&E overload, invest in staff who can improve the flow through beds). Unfortunately the first step would be to develop an analysis of how the whole system fits together and therefore identify which incremental investments would most improve the productivity of the system. There is no such analysis, though there are plenty of hints that workforce isn't the biggest problem. 

The workforce model as currently discussed seems likely to further distract the NHS from other major problems. A focus on the front line risks distracting from even bigger staff shortages behind the front line. And from a long term neglect of capital spending (leading to major issues with buildings, equipment, and IT). Furthermore there is a big risk that a long term workforce model could ignore the short term decisions that might help the immediate problems with workforce and could encourage the NHS to build in false assumptions that fossilise bad current choices.

That's a lot of risks for an unclear outcome. Whatever the intuitive attraction of a transparent model of workforce needs, it is far from obvious that the NHS would get anything useful.

We need a more informed debate about what holds back NHS productivity, not a model focussed on the front line workforce.

Nostalgia for face to face appointments should not distract GPs from real choices they have to make

[NB this was submitted to the BMJ as a possible blog in March 2021. They rejected it as not topical enough. But the relevance of the topic has increased a great deal in April and May as GPs have seen record workloads. So I thought i'd post it here to to remind me what I was thinking then before I write about the latest developments in the debate.]

There is a lively debate on how GPs should work once the pandemic is over. Many have adopted what NHSE calls "total triage" where patient needs are assessed before booking any appointments and as many requests as possible are handled remotely (online or by phone).

Should they stick with this approach or go back to seeing most patients face to face?

Examples of the debate are this lecture by Kath Checkland titled Who needs to see a doctor face-to-face? Or this Guardian article where Martin Marshall claims most patients prefer face to face (f-to-f) appointments.

The trigger for the debate appears to be the major drive by NHSE to get GPs to adopt non f-to-f ways to handle patient requests during the pandemic. 

But the discussion of the value of f-to-f appointments risks distracting GPs from the reality they have faced and will face again. Sure, there is something special about f-to-f appointments–GPs can pick up body language cues and building a relationship with patients is easier–but this does not imply that every request needs f-to-f or that the tradeoffs achieving high-levels of f-to-f appointments are worth making.

The pre-pandemic reality for most GPs was that the majority of patients did not get a same day response to their requests and the average wait for an f-to-f was more than a week. Is that time tradeoff really worth it for the benefits of a person to person meeting? Worse, the reality of most practices was that access didn't depend on patient need but on their ability to get through a phone lottery at 8am (my GP is still like that: getting a call back on the same day is possible but required me to make 30 attempts over a 1 hour period to get through on the phone at all). 

The world where the default response to every request was a same day face to face meeting has not existed for decades.

The reality is that GPs need to balance the supposed benefits of f-to-f against the tradeoffs. And not just the tradeoffs they desire, but also the ones their patients want and need.

This is where we have to test the attributes of different online triage tools. 

The couple of dozen tools available are supposed to make getting and triaging patient requests easier for the patient and the GP by moving as much as possible online. The major benefit of this is to make the internal flow of work inside the practice much more efficient (but many of the available tools are quite late to that party). This frees capacity, enables the ability to triage based on need, allows more flexible responses based on need and, as a result, improves the speed of response. 

Many object that moving patients online disadvantages the old and less tech literate. But this is only true if being online is compulsory. Good practices don't close down incoming phone calls or walk-ins. Good online tools allow the practice to process phone requests internally using the same tool that facilitates those requests arriving online (again, though, many do not, creating a very different experience for those using the phone).

Others object to the need for online systems to classify the patient request into simple boxes. Patients, as Checkland pointed out, often have vague amorphous requests rather than simple classifiable problems. But, again, this is a straw man that only applies to those systems that insist on asking 20 questions to try to reach a diagnosis before the GP sees the request. Not all systems work like that: some allow a simple free text request to be submitted which a human can assess and triage before deciding how to respond. 

What good online triage systems offer is an easier way for patients to get their request in front of the GP. And they enable the GP to offer a faster, more flexible range of responses (including bringing the patient in to be assessed in person). If anything online tools make getting f-to-f meetings easier for the patients who need them than the traditional phone lottery. 

But they also reveal some truths the fetish for f-to-f obscures. First the majority of patients don't even want f-to-f appointments if a faster useful response is the alternative. Second, patients like being offered a choice of response mode. Third, GPs can be far more effective if they adopt flexible responses as many of these take less time than f-to-f and free up time for the most needy.

The widespread nostalgia for f-to-f obscures all the tradeoffs uncovered by online tools. I'm not even sure that, if we had twice as many GPs who had the capacity to do same day f-to-fs for all, that this would be what patients preferred. But, since we live in the real world, GPs need to judge whether doing far more f-to-fs is worth the tradeoffs in time and convenience for patients. If some patients really need f-to-f assessment, how are you going to identify and prioritise them? Pre-pandemic practice often traded high proportions of f-to-f for long delays and prioritised by lottery not need. 

So what should GPs do? 

• Don't reject online tools because they think they are inimical to good f-to-f practice. Perhaps–even if they love the benefits of their f-to-f meetings–they should choose online triage tools that make it easy for them to identify the patients who will benefit most from f-to-f. 

• Choose tools that ask patients what they want and don't assume everyone wants f-to-f.

• Choose tools that patients like, not ones they won't use. 

• Monitor the results: monitor what patients want and whether they get it; Monitor their satisfaction; check that triage works effectively.

Certainly don't pine for the non-existent days when f-to-f was the perfect solution to everything with no tradeoff in time or convenience.

 

How busy are English GPs?

Our GPs are probably grossly overworked. There is plenty of evidence that this is true, but recent data collected by NHS Digital paints an ambiguous picture. Many GPs have reacted to my analysis of that data with incredulity, claiming NHS Digital don't know what they are doing. Or that the data is meaningless and too much of a burden to collect. The real situation is more complex and won't be fixed by NHS Digital alone but requires GPs to pay more attention to how they collect data and why it is collected. And it is critically important that the system does a better job of collecting data about their activity or the case for higher primary care funding will fail as soon as it is examined by the treasury. GPs, their system providers and NHS Digital all need to work together to create more useful data.

Recently released data poses some interesting questions for GPs

At the start of 2019, NHS Digital released the first public version of a new dataset on GP activity they had been collecting since the end of 2017 (and they are currently updating monthly). This dataset summarizes the number of consultations each day using data collected directly from the (major) providers of GP clinical systems (not all of the minor providers are covered yet). 

The public version of the dataset covers the CCG-level aggregate number of appointments and the status of those appointments in several different categories:

• The appointment mode (face to face, telephone, visit, video etc.)
• The appointment status (basically whether the patient turned up for a booked slot or not)
• The type of staff who saw the patient (essentially whether the person who saw the patient was a GP or someone else)
• The delay between the booking and when the appointment actually happened

The data coverage is good for most CCGs (only one has no data at all) and there are some big, interesting insights in the data. For example, the number of no-shows ("did not attend" or DNAs) is very strongly related to the length of time a patient has to wait for an appointment (see my analysis here and a discussion in the BMJ here including comments on why the results are different to some previous academic analyses of DNA causes).

But the analysis I did on the number of appointments per day done by GPs caused a much bigger kerfuffle. I want to be fair and present those criticisms alongside the analysis. It is possible that both the analysis and GPs criticisms of it are right, but, if that is correct, then GPs, their systems providers and NHS Digital need to make some significant changes to how they work.

Before we start, a thought experiment

To put the results in context, it is worth doing a simple thought experiment to put the results in perspective.

Imagine a GP who does nothing other than see patients. She fills her entire 8-hour working day with 10-minute appointments with no breaks. Let's also assume that she works 48 weeks out of 52 (including bank holidays in the 4 weeks of downtime). And let's assume that each day worked consists of 48 appointments (this is how many fit into an 8-hour day if she takes no breaks between appointments or for lunch and does any other work after hours or at weekends). That means she does 11,520 appointments per year or about 44 per weekday in the year (she doesn't work every weekday because of holidays).

That 44 appointments per day doesn't sound like a sustainable workload. So let's assume a 1hr lunch break and gaps between appointments that brings the average work done down to 5 appointments per hour. Now she does 8,400 appointments per year or about 32 per weekday averaged over the year. That's probably still not sustainable every year but it is a thought experiment and we can make the simplifying assumption that GPs are superhuman. Bear this number in mind when we look at the actual reported workload from the NHS Digital data.

And, don't forget, that a real GP also has a ton of paperwork to process plus they need time for professional development, time to train new GPs and other staff and time to manage the practice. So lots of other work to fit into the actual working day alongside the 8-hr shift spent actually doing appointments. And, remember this analysis simply takes the total number of appointments recorded in a CCG divided by the number of weekdays and the number of FTE GPs; it ignores all the other work and all the extra time GPs put in beyond normal hours or at weekends.

What does the NHS Digital data show?

The original claim that prompted me to look more closely at the NHS Digital data was a simple calculation about the number of appointments per GP based on the headline numbers. NHS Digital estimate that about 307m appointments were recorded in the year to April (they adjust the total for missing data and their adjustment seems to differ from mine by about 10m/year but this is not a huge gap given other uncertainties in the data). Taking their number per working day in the year and adding the March estimate that there are 33,425 FTE GPs in England suggests that the average GP does about 35 appointments per working day. This seems to confirm, when compared to my thought experiment above, that the typical workload is well into unsustainable territory with a side order of "if we keep this up we are all going to die of overwork".

But that isn't what the dataset actually says. GPs have a lot of other staff who handle >40% of the patient contacts according to the data (this is the national average but see the map below for the variation among different CCGs in the % seen by non-GPs).

If you take that into account, the data actually says the average GP does something closer to 20 appointments per day. 

20 appointments/day is the England average. But, usefully, we have the staff census data that tells us how many FTE GPs there are in each CCG and the appointment counts are available for each CCG as well. So I can do a CCG-level analysis and look at the variation across the country (taking into account the number of GPs in the CCG and the % of appointments with a GP). If we generously assume that activity where the staff type for the staff fulfilling the appointment is "unknown" is done by GPs we get the following distribution:

(Note: each block on this chart is a single CCG and the activity data is based on activity done on weekdays in the first 4 months of 2019. Also note that an interactive version of the dataset and many analyses is available here and you can test how varying some of my assumptions changes the results.)

In about 8 CCGs the GPs appear to be seeing an average of about 26-30 patients per day. The commonest CCG average is, however, closer to 20/day. Also, don't forget, the results are averaged over at least a month and reflect the average activity on all available days, not just the busy days where all the GP time is devoted to seeing patients.

Don't forget, though, that this is the average across a whole CCG: the variation among practices within each CCG is probably large so there will probably be some practices where the GPs do >30/day even when the CCG average is just 20.

Some GPs were indignant when I put an early version of this chart on Twitter. The typical response was "20 a day! I do far more work than that in a typical day. NHS Digital don't know what they are doing. The data is obviously rubbish…". I'm not entirely convinced that all the criticisms are fair, but it is worth looking at some of them to see if there are lessons to be learned.

Possible problems with the data

This just doesn't reflect my typical day

This could be because the data is just wrong, but we will come to that possibility later.

Another possible explanation is that what GPs remember about how busy they are isn't an accurate reflection of the average number of appointments across the whole month or year. If, for example, what GPs remember are the day when they are focussed on seeing patients but they also devote other days to practice management, paperwork, training or some other work that doesn't involve booked appointments, then their memory won't match the averages reported here: the results will reflect the total appointments divided by the total available time for all work not just the typical day when the GP is dedicated to seeing patients. 

This work (and my initial thought experiment) assumes a 40hr week. Many GPs spend a lot more than 8hrs a day working (perhaps doing all the admin in their unpaid overtime). This dataset is not recording that work or adjusting the available hours to account for it. This doesn't impact the results but it may make the perception of the results much less clear than they should be.

The data clearly is missing a lot of the work we do

Part of this is a perception problem as discussed above. But it could clearly be true that a lot of activity is not correctly recorded. 

But the blame here isn't on NHS Digital. The data is directly collected from the major clinical systems used in practices to record activity. If the numbers are a big understatement of the appointments being booked, then practices have a big problem not just NHS Digital. I suspect we are not understating the true number of appointments by a large margin.

But we could be understanding the amount of activity. Many patient contacts don't result in a booked face-to-face appointment. GPs do some work online or by telephone. Clinical systems are not uniformly good at recording that activity as I know from comparing incoming requests via askmyGP (an online tool that manages all incoming patient requests and helps GPs to completely change their workflow when responding to those requests in ways that improve the speed of responses to patients and lowers the degree of overwork for GPs). Successful online tools (and some phone-triage approaches) often result in only 1 in 3 requests leading to a face-to-face appointment. This reduces the number of appointments recorded in the clinical system while providing satisfactory responses to the other patients. But much of the activity for those other responses may never appear in the clinical systems. As GPs adopt new ways of working their clinical systems may become a lot less good at measuring their activity.

On the other hand, the NHS Digital data does record a lot of phone activity. Between 10 and 15% of all appointments are recorded as taking place online, by phone or by video. We know that recording of this is patchy, but so is uptake of alternative ways of responding to patient demand. Some GPs still think that face-to-face appointments are the only response despite mounting evidence that offering a wider range of alternatives can reduce GP workload and make patients happier because they get a faster response.

We have to admit, though, that traditional GP systems have not caught up with this change in practice and won't consistently record the shift in activity reliably.

NHS Digital will draw all the wrong conclusions from these results

I got the impression from some comments on my initial analysis that, not too exaggerate too much, central NHS bodies are just a bunch of malingering bureaucrats who exist just to malign hard working GPs by misrepresenting how hard they work by providing the NHS leadership with false, irrelevant data about what they do. And that any suggestion that this should be fixed by GPs spending more time to provide accurate data about what they do would be yet another straw added to the already broken camel's back that is the typical GPs unsustainable workload (or some such mixed metaphor for even more overwork).

But NHS Digital's motivation is not to burden GPs with more work or to undermine the case for more GPs. It is to understand what GPs are doing to develop better policy, and that can't be done without better data. Originally the idea was focussed on developing a better understanding of winter pressures but it should be useful for bolstering the case for more investment in primary care or for promoting better working processes. 

But, imagine the conversation the NHS would have with the treasury when bidding for the money to recruit another 5,000 GPs:

NHS "We need another 5,000 GPs."

HMRC "How much will that cost?"

NHS "A couple of billion pounds a year."

HMRC "That sounds like a lot. Why do you need it?"

NHS "They keep telling us they are overworked."

HMRC "That's what everyone says. How much work do they do? Show me some numbers."

NHS "They say they are doing unsustainable numbers of appointments a day."

HMRC "How many is that."

NHS "We don't know exactly but more than 30/day doesn't sound sustainable"

HMRC "So how many are they actually doing?"

NHS "No idea. But I'm sure it is a lot. We really need the money to recruit more GPs."

HMRC "Sure. You can have it when we cash the £350m/week we will save from Brexit."

NHS "Really?"

HMRC "No." 

HMRC "We were joking. Sod off and don't come back until you have some actual evidence."

OK, I'm parodying the case for more money for GPs and there are other hard sources of evidence that say they are overworked. But if the NHS as a whole doesn't understand what, in detail, GPs are actually doing any case for reform or extra cash is going to look weak.

There are more reasons than this to have reliable data. GPs need it to understand their own work or they won't have any idea how to do a better job. While the NHS Digital appointment data doesn't capture everything they do, it is at least a start providing a better understanding of a large part of their activity. And the main limitations that prevent the data being more useful are not NHS Digital's fault but the fault of the system providers and choices made by GP practices. 

The data isn't consistent or comprehensive

GP Clinical systems are built around a model of GP work that assumes everything revolves around a pre-booked face-to-face appointment. GPs can vary how they label the slots for activity in the system and they don't always assume every appointment is 10mins long (though their royal college doesn't seem to have caught up with this realisation). Systems usually allow the exact time taken for each appointment to be recorded (assuming GPs press the right buttons which they don't always do). Some systems make it easy to record activity in open-access clinics. But systems tend to assume every appointment slot is the same length and some are completely, irretrievably hopeless at measuring the time taken for consultations done by phone. 

One of the biggest problems faced by NHS Digital is the number of different descriptions given as labels for each slot. Practices are, essentially, free to label each slot however they want and one estimate says that there are tens of thousands of different labels in use across practices in England. So when NHS Digital try to acquire data from everyone it is a gargantuan task to correctly identify the different types of appointment and group them together in ways that make sense. Was that a face to face pre-booked appointment? Or an ad-hoc on the day f-to-f as part of a walk-in clinic. Or a slot where a nurse did flu-jabs?

And, as more activity is initiated online and GPs choose to triage patient requests before booking appointments, there are more varied ways for GPs to respond to patients. And the systems–designed as they are around the idea that most activity involves pre-booked slots–don't record all those interactions. In practices using online tools like askmyGP only around ⅓ of requests result in a face-to-face appointment (e-Consult, another online tool provider estimates a similar ratio). This means, potentially, that the majority of GP activity won't be recorded in the traditional clinical systems.

So it may well be true that what NHS Digital are collecting is not a reliable guide to what activity GPs are doing. But the fault can't be fixed by them: if we want better data the problem has to be addressed by the practices (who need more consistency is how they use their systems) and the suppliers (who need to design their systems to break the model that everything neatly fits into 10 or 15min slots).

Some suggestions about how to do a better job

It is not at all helpful to complain that the NHS Digital GP data is rubbish. If we want to do a better job in primary care (to improve how GPs work and make an unanswerable case for more investment) we need better data. 

An article in the Economist describes some improvements in primary care and how they were done:

When the four practices serving St Austell merged in 2015, it was an opportunity to reconsider how they did things. The GPs kept a diary, noting precisely what they got up to during the day. It turned out that lots could be done by others: administrators could take care of some communication with hospitals, physios could see people with bad backs and psychiatric nurses those with anxiety. So now they do. Only patients with the most complicated or urgent problems make it to a doctor.

The key point here is that only when they had reliable data about their activity could the GPs redesign the work they did in ways that would improve it.

As a recent Nuffield Trust comment on the GP dataset says:

In hospitals, routine datasets such as Hospital Episode Statistics (HES) give us insight into who gets treated, for what, by whom, and where. This basic knowledge of the activities and performance of hospitals is now essential for making evidence-based policy changes, monitoring existing policies and providing fundamental administrative data to run the system and allocate funds. There is a clear need for something similar in general practice…

But, they continue:

NHS Digital makes it clear that what has been released so far still needs improvement. As it stands, there are a number of problems with the data and these limit the extent to which we can use it as a resource for evidence-based policy. 

Having different practice IT systems gives GPs flexibility over the way they organise their work. But the absence of reporting standards in general practice means information is recorded differently in each system, making it difficult to combine the data, and information gets lost.

In the early 2000s, hospital activity data (HES) was in a similar state of disorder. Doctors took little interest in making sure that the amount of activity recorded was correct or the clinical details correctly coded. They too argued that this was just an unnecessary bureaucratic task required puerly for "feeding the machine" (even though the most basic metric of hospital success–not killing your patients too often–requires accurate coding). 

But HES got a lot better. One way it got better was to show medics what HES said about the work they did. The Royal College of Physicians ran some interesting experiments starting in 2002 (reported here and here). They took HES data and fed it back it to the clinicians whose activity it recorded. In one of the reports on this exercise they argue this:

A vicious circle ensues: routinely collected data is perceived as being of poor quality and unable to support the needs of the individual. Individual clinicians avoid the use of such readily available information... Centrally held datasets remain unchanged through neglect, clinicians failing to engage with the information process in their trusts and remaining ill at ease with the records of activity which result. It is clear that if this cycle is to be broken, steps must be taken to engage clinicians at a level whereby the information is made readily available, accessible in format and of use to clinical practice. By examining routine data from a clinical perspective and feeding issues of quality back to trust information departments the cycle can be reversed.

Hospital clinicians being shown their own data in a digestible form was a big part of the drive that improved the quality and utility of HES data. GPs and NHS Digital need to do something similar.

Perhaps NHS Digital could take the first step by making practice level data from the central collection available to practices in a useful, comprehensible format so GPs can see what the data actually thinks they did. This would be the first step in driving improvement in what is recorded.

Conclusion

In short, here are some simple ways for the different players in primary care to change what they do in ways that would promote real improvement for GPs and their patients.

NHS Digital should make a friendly, easy to comprehend, version of their dataset available to every individual practice so GPs can compare what they think they did to what NHS Digital's extract has recorded.

GP System Providers should make their systems more flexible in recording non-appointment activity and should work to support GPs to use more consistency in how they record and label their activity.

GPs and practices should not just dismiss the data as being unreliable and useless. They should strive to understand it and seek ways to work with NHS Digital to make the data useful for local purposes and more reliable for policy making.

Everyone should not dismiss data they don't like but engage with it and seek to make it both more reliable and more useful.