Last week I fired off a rant about how politicians misuse statistics to win arguments with little regard for the relevance or context of the numbers thereby doing extreme violence to honesty and truth telling. This week I find yet more examples from the NHS.
Here is why the specific instances constitute bullshit.
The Prime Minister said the following in Prime Minister's Questions on January 24 in response to a Jeremy Corbyn complaint about poor performance in the English NHS over winter:
If he wants to talk about figures and about targets being missed, yes, the latest figures show that, in England, 497 people were waiting more than 12 hours, but the latest figures also show that, under the Labour Government in Wales, 3,741 people were waiting more than 12 hours.
Corbyn didn't spot the problem (I presume neither he nor his advisors are any more knowledgeable about these statistics than the PM or her health minister).
But it is quite a simple issue: the Welsh NHS counts something different when it measures 12hr A&E waits than the English NHS. In Wales, the clock starts when the patient arrives; in England the clock starts when a decision is made to admit the patient to a bed. This decision is highly gameable and is highly gamed. Even without any explicit fiddling of the numbers (there are rumours that some management teams stop their teams recording a time in a timely way to reduce their reported numbers) the decision is often postponed until the hospital knows there is a bed available. This may happen after the patient has already waited 12 hours in A&E.
The English "trolley wait" metric is a terrible, useless and misleading metric. It actively distracts from a good understanding of the problem of long A&E waits. Yet here we have a politician using it to win an argument with the opposition instead of trying to understand what is going on in A&E.
Here is some help to put it in context. The comparable number is accessible from data collected from hospitals (it is trivial to calculate true 12hr waits from A&E HES data, it just isn't routinely done and won't yet be available for this winter as national HES takes a few months to compile). In january 2017 there were 46,413 true 12hr waits in English A&Es (these figures were release by NHS Digital after an FOI request). That is the comparable number May should have quoted (if she had an up to date version of it which we have no reason at all to assume would be better in January 2018). If anyone in the system cared to have reliable and useful numbers to tell them how A&E was performing, they could easily collect these numbers on the same basis as Wales enabling them to have a much better and ungameable insight into what is really happening. Guess why they don't do that.
The disease, unfortunately, runs deep. Here are some extracts from Pauline Phillip's report on winter pressures to the NHS Improvement board on january 24:
...management information for January suggests an improvement and the system is performing better than at the same point last year…
[compare to this a few paragraphs later]
...A&E performance for December was 85.1%. This is 3.8ppt below the previous month (88.9%) and 1.0ppt lower than the same time last year...
...Performance is impacted by higher bed occupancy than last year and increases in attendances and emergency admissions...
[a few paragraphs later]
...Type 1 attendance growth compared to the same month last year is 1.0%.
[a 1% increase in attendance is actually below the long term trend in increases, though, to be fair, admissions were up by a much larger amount and they matter more. OTOH the excuse that "performance is impacted by...increases in attendances" is not the most accurate way to report the situation]
...the trend for much lower long trolley waits continues; 12 hour waits are 10% lower compared to the same time last year.
The problem here is that the text is frequently misleading when compared to the numbers quoted (always a danger when people are allowed to write paragraphs of bullshit instead of showing the clearest analysis of the key data points in context). And Philip, who should know better having run one of the hospitals with an outstanding A&E department) goes on to use the same trolley wait statistic that the PM quoted in her answer to make a claim that things are improving. It is such an unreliable statistic it tells us no such thing.
If you are going to manage A&E better you need to use numbers that are reliable indicators of what is really going on. Not terribly misleading numbers that are both a poor, misleading metric and utterly gameable. The trolley wait metric should have been burned years ago.
Maybe this point is simply not understood and the politicians and NHS leaders just don't get that this statistic is bullshit. Maybe this interpretation lowers their culpability for promulgating bullshit, but it is hardly comforting that the people in charge of improvement don't seem to possess the basic knowledge that any competent analyst of A&E statistics has known for years.
The reason why England doesn't routinely release reliable numbers about long waits in A&E is that they are very embarrassing. If they were widely used by NHS Improvement, as they should be, to understand what was really happening so their efforts could be focussed on generating real improvements, there would be a lot of bad headlines (which might be worth it if it helped but led to actual improvement).
Sadly, in politics and organisations dominated by political management, improvement isn't the point: good headlines are all that matters. The impact of political bullshit is pervasive and corrosive.
PS. I'm not the only one who noticed. Faye Kirkland posted this on twitter just after I completed the original version of the blog. It is a letter to the Chair of the UK Statistics Authority pointing out just how misleading her comparison was. It will be interesting to see how he reacts.
If governments want to make good decisions they have to have reliable data about what is happening. But they increasingly don't use numbers that way. Instead of using data for insight they use it for bullshit and undermine the evidence they need to make a difference to anything.
So the NHS is having a winter crisis. This year instead of the service responding in a panic when the unpredictable event of winter occured, the panic response was, apparently, planned. Apparently, this is good, our lords and masters said so.
But there is a little vignette that occured in Parliament that illustrates a great deal about why we have such problems and even more about the reasons why we currently look like a kakistocracy. It relates to the statistics about bed occupancy in hospitals and illustrates something profoundly disturbing about how politicians handle statistics and use numbers.
The background to the story is that the government has now mandated daily statistics about "winter pressures" in the NHS. That might not be a bad thing in itself if the point were to make management decisions in response to the numbers (though this supposes an ability to know what response to make and to interpret the numbers correctly: neither are obviously true).
One of those statistics is bed occupancy. This isn't a very useful statistic (as I've argued before) but collecting it daily is much better than weekly or monthly which is what is done for the rest of the year.
The government (and many others) have set a "target" level of occupancy for beds to ensure there are enough free beds each day to cope with demand. That target says no more than 85% of beds should be occupied.
So far so good. But the annoying doctors and opposition insist we are in the middle of a crisis in bed availability and keep complaining. In response to one of those complaints and in explaining the impact of his winter plan Jeremy Hunt said this in the House of Commons:
The shadow Health Secretary told The Independent: “It is completely unacceptable that the 85% bed occupancy target…has been missed”. What was bed occupancy on Christmas eve? It was 84.2%, so this had a real impact.
To put his claim in context, here is the chart of daily occupancy to early january (from the latest data I could get from NHS England). Shading identifies complete weeks:
Which number did Jeremy Hunt repeat? The least representative number on the chart and the only day in the whole of winter where the target was met. He also ignored the longer term context. The days near christmas have the lowest occupancy of the whole year and, historically, have often been in the 60% range.
Now maybe he was just having a bad day and didn't mean to quote something so irrelevant to the current problems with beds. But another minister said this two days earlier when challenged with a similar complaint in the Lords:
The noble Baroness talked about bed occupancy. Of course, we know that high levels of bed occupancy are a concern. Bed occupancy was below the target of 85% going into this period—on Christmas Eve it was 84.2%
I think we can conclude that this number has been shared around the government as the one to quote to deflect any complaints about the state of the NHS at winter.
Sure politicians have to win debates and this will, inevitably, involve some spin. But the way this number was brought up goes beyond reasonable spin and becomes what Frankfurt would describe as bullshit:
[the bullshitter] does not reject the authority of the truth, as the liar does, and oppose himself to it. He pays no attention to it at all. By virtue of this, bullshit is a greater enemy of the truth than lies are.
What this case illustrates is a deeply troubling view about how politicians treat statistics. They do not look to them as a source of useful information they can use to make decisions. They trawl them for any number that supports the argument they are trying to make, regardless of meaning or context. In doing this they utterly devalue their use in decision making or management.
There is an alternative explanation that is slightly less pejorative: perhaps they are so statistically illiterate they don't understand the numbers or the context. Unfortunately it isn't obvious that this explanation bodes any more optimism about the how well the country is run.
I tell this story as an illustration of a very widespread and pervasive phenomenon in modern politics. It isn't just the government, they are all at it, opposition and minor parties included. There seems to be no drive to take the effort to analyse problems before deciding key policies or actions. The process now seems to be to identify some actions or policies likely to play well in newspaper headlines or with supporters. Only then, after the key decisions are made, does anyone look at the evidence and then only to wrench some number, no matter how out of context or irrelevant, that supports their view. Even when the number is rebutterd by the highest statistical authority in the land, they will often continue to quote it (as Boris has just done with the legendarily bad £350m/week goes to the EU). Truth and context are irrelevant: all that matters is winning the argument.
This is no way to run a government. We need people in government and opposition who are competent, honest and who are prepared to do the hard work of analysis before making arguments or deciding policies. If we don't get them, and get them soon, the bullshit will overwhelm our ability to make any good decisions about anything in public policy.
The NHS needs more money. But the belief that it needs less management or administration is nonsense. It won't spend any new money well unless it improves its ability to spend that money well. That means it needs more management not less.
The FT has a well-deserved reputation for balanced and factual commentary on the big issues. So I was surprised to see this cliche repeated in an editorial on January 5: "There are too many administrators and not enough front-line medical staff."
Other commentators constantly repeat similar untrue cliches. On Radio 5's "good week bad week" on Sunday I heard someone claim "the NHS has more managers than nurses".
It isn't true. The reality is that the NHS is one of the most undermanaged organisations on the planet. Here are the numbers from the NHS staffing system.
There are ten times more nurses than managers and three times more doctors than managers.
And the number of managers has been falling. The numbers were cut by about 30% by the Lansley reforms because he believed the cliche ("more resources to the front line" which I've argued before is one of the stupidest in the debate on NHS policy). It hasn't obviously worked.
Just for reference here is the relative numbers of different types of staff compared to their levels before the coalition government took over in 2010:
What is notable is the steady and then sharp decline in manager numbers (with a subsequent slow increase as the system realised it had drastically overdone the cuts). Also notable is that consultant numbers are rising a lot but nursing number are steady (which suggests that recent complaint that medical productivity is limited not by doctor numbers but by the lack of nurses and support staff has significant support in the actual data).
When the bill was being debated and proposed a sharp cut in manager numbers, I tried to find some benchmarks for how many managers an organisation like the NHS might need. One crude one would be to compare the NHS against other organisations in the UK. Unfortunately the ONS only counts managers in the economy as whole and not by industry or sector (about 11% of the workforce are managers according to them). So I looked elsewhere (see the original BMJ letter reporting this here and the longer version here).
In the USA charities have to declare how much of their budgets are spent on three separate categories: money spent fundraising, money spent on their projects and money spent on deciding how the run the charity and allocate their spend. That last category is the one that might help us estimate the money an the organisation spends on management. It isn't a perfect proxy but it isn't bad. Charities, like the NHS, are not in the business of enriching their chief executives and they are under pressure from supporters and regulators to be frugal so as much of their spend should go on their purpose not on overheads. But frugality has to be balanced by the need to spend money well. Spending too little on good decisions is just as bad as spending too much.
Most charities spend more than then the NHS; medical charities often spend 3 time the proportion of revenue as the NHS. If the NHS were a charity they would risk investigation by regulators for a lack of management capacity.
There is one important caveat to this analysis. When I was looking for benchmarks I was focussing on the very heavy cuts to managers in commissioning (this was the focus of the Lansley cuts). CCGs and national bodies are the groups responsible for deciding how to configure services across the country or in a particular area. They are the people who have to decide whether it might be better to spend more in the community and less in hospitals (which traditionally dominate everything in the NHS). If they don't have the capacity to make good decisions, then the NHS is in trouble as it will be stuck with the way things currently are whether that is good for the population or not. The charity vs commissioning analysis is particularly stark for commissioners who now have so little management capacity it is hard to see how they get anything done by themselves (this, perhaps, explains their extensive use of management consultants which is often complained about by people who don't seem to understand the lack of management capacity that drives it).
But the lack of management in hospitals is also a problem. Manager jobs there should be to design effective systems, to coordinate the work of front-line staff and to do the analysis that drives and sustains improvement. This should lower the burden of paperwork and admin on doctors and nurses. If there are too few managers doing the right things then bad and inefficient processes will persist, lowering the quality and productivity of all the work done by front line staff. Improvement won't happen. And the doctors and nurses will spend too much of their time on administration instead of treating patients. IT is very obvious from the overall staffing numbers that hospitals, not just commissioners, have far too few managers.
This should be more obvious than it seems. The big NHS problems are problems of coordination and operational effectiveness. The NHS has a big issue with knowing where to spend money to make the whole system better and struggles to consistently improve or to spread best practices quickly. These are managerial in any organisation and management failure or lack of capacity makes extra spending, even when it arrives, a lot less effective than it should be. If you just spend more without knowing where the big problems are, you may well not improve the problems at all. This is abundantly illustrated by the persistent failure to analyse the real reasons for the decline in A&E performance (see my analysis).
Weak management also leaves the service incapable of resisting stupid ideas coming from the political centre. For example, Jeremy Hunt's proposal to put GPs at the front door of all A&Es is an idea that any competent analyst or manager would resist because it couldn't possibly work. Lack of management capacity leaves front-line nurses and doctors working with badly designed processes and many end up spending far too much time on administration when they should be treating patients.
There is plenty of evidence that the NHS needs more money. But even if the extra money arrives it will yield far fewer improvements than it should if the people spending it are short of management capacity. It is time to kill they myth that the NHS is overmanaged. In fact a lack of management capacity is one of its biggest problems.
