Thursday, 21 April 2016

Gresham's Law works in health policy: bullshit squeezes out honest truthful analysis

Gresham's law is an economic idea that states that counterfeit money drives real money out of circulation. The same thing seems to happen in healthcare policy: good analysis of problems and good policy is squeezed out of the debate by bad analysis and bad policy that sounds good but won't work. This tendency has to be fought vigorously or it will become impossible to improve the NHS except by accident.
"Bullshit is a greater enemy of truth than lies are…" Tim Harford
Gresham's law is a very old principle in economics (it has been known since Aristophanes) which states that bad currency drives good currency out of circulation. I won't say more about the economic mechanisms here as this is an article about health policy where I've noticed that the same sort of problem appears to be happening there.

In short, dumb analysis of what the problems in the NHS are is starting to dominate intelligent, reliable analysis in debate and in policy making; dumb policies are driving out the good ideas that might make things better.
There are a number of reasons for this problem. One was dissected masterfully by Tim Harford in a recent FT article:
This is the real tragedy. It’s not that politicians spin things their way — of course they do. That is politics. It’s that politicians have grown so used to misusing numbers as weapons that they have forgotten that used properly, they are tools.
But we should not rush to blame politicians for dumb analysis of problems in the NHS. Many in the commentariat and many workers in the NHS are keen to shortcircuit good focused analysis and substitute attractive but dumb policy ideas.
There are, unfortunately, a large number of plausible analyses of key problems are that are just wrong. And, if you leap from bad analysis to policy, an equal number of solutions that sound good but will simply waste time and resources because they won’t work. (I would say obviously won’t work but that is apparent only to those who do the detailed, dirty work of actual analysis and the bad ideas are often attractive only because nobody has stopped to do any actual analysis or because nobody has paid any attention to the analysis that has been done). Leaping from symptoms to treatment with no intervening effort on generating a correct diagnosis is bad in medicine and just as bad in management.
One major cause of the Gresham effect here is that many proposed solutions sound good. So they take up space in newspaper headlines and discussion and thereby exclude the more nuanced solutions that require some explanation of why things are a bit more complicated than that. Hence Jeremy Hunt’s repeated assertion that death rates are higher at the weekend. It is a nice, simple idea that sounds plausible and backs up one of his favourite policies: a 7-day NHS. But the reality is the analysis is complex; we probably can’t be certain that the mortality really is worse at the weekend; and we certainly don’t know what causes it even if it is true. So using it as a crutch to support an attractive policy (who doesn’t want the NHS to work the same at weekends?) is deeply misleading. It is particularly misleading because even if we need an NHS that works the same way at the weekend it is far from obvious that changing doctor’s contracts will make any difference. We have plenty of operational evidence that the NHS doesn't work well at weekends but it doesn't point the finger at medical staff as the key problem.
Another zombie idea that wasted space in policy and newspaper headlines was the idea that problems in A&E were caused by changes in the GP out-of-hours contract. Superficially it looked like the numbers attending A&E grew strongly after the contract was changed. But that was coincidence: England started counting attendance at minor injury units (and the number of such units grew rapidly) around the same time as the GP contract was altered. Core attendance at major A&Es (which is where all the problems with treatment speed are) didn’t change from its long term trend. And those who know the statistics also pointed out that few people attend A&E at night; volume is far higher during the day and peaks when GPs are still open.
In fact policy about A&E is littered with dumb ideas that simply can’t be reconciled with the actual data. The idea that A&E performance is declining because of too many people turning up is attractive. So there are repeated discussions about policies to respond to this: diverting patients somewhere else; massively increasing staffing in A&E; putting GPs at the front door… They all sound like they might do something. But every dataset we have says the performance problems have nothing to do with volume. In fact the best analysis says the biggest cause of slow A&Es is nothing to do with the A&E department at all: it is about the inability of hospital wards to accommodate the flow from A&E admissions. Spend all you want on the other policy ideas, but, if you ignore that bottleneck, you are wasting your money. Sadly, every time A&E performance deteriorates, we get a torrent of bullshit policies and almost no commentary that tries to identify (or points out that we have already identified) the most important problem and can do something about it.
One of the most important areas where bad ideas squeeze good ones from the arena is money. The NHS as a whole could probably use more money and probably should get it. Many parts of the system have been campaigning to get more of the budget for their activity. GPs complain that their share of the NHS budget has been falling (and then describe this as "cuts" when it isn't). They claim they are swamped by patient demand and can't cope without vastly more investment. The problem here typifies the way bad ideas squeeze out good ones. The bad idea is that all the problems are caused by lack of (or will be solved by more) money. Nothing else matters. There is therefore almost no discussion of whether anything other than an increased budget could make the life of a GP better or help the GP do a better job for patients.

Yet there are concrete examples that show GPs who pay attention to how they match their capacity to the things patients actually want (the demand) can dramatically lower their workload at the same time as improving patient satisfaction. Flexible attitudes to how patients needs are met and wider use of modern technology can more than fill the perceived gap in capacity that GPs campaign about. But this gets almost no attention in the debate as operational ideas that work are squeezed from the arena by demands for more money.

The idea that the only problem is money is insidiously dangerous across the whole NHS. I'm sure the system could do better with more. But to focus on campaigning for more money and forget all the other things that could be done to improve things is disastrous for several reasons. One is that getting more money is unrealistic in the short term; we should be seeking improvements that can make a difference right now. Another is that getting more money without fixing some of the current problems is a likely to guarantee that the money will deliver far less benefit than expected should it ever arrive. If we lack good management systems that create an awareness of where the real problems are, we will spend extra money on things that don't address the problems and have little impact on the actual problem. A belief that money is the only problem pushes out any thinking on the problems we could fix right now without any extra money and prevents us acting now so we spend any future money on the areas that will yield the largest benefits.

Another attractively populist idea is encapsulated in the slogan "more resources to the front line". It is attractive because it makes a good slogan. It feeds the popular myth that bureaucracy consumes too much money for no useful purpose. It panders to the idea that every problem is solved by having more front-line staff. Sadly every analysis suggests the NHS is extraordinarily undermanaged (see my comments here). While it is possible to be too bureaucratic and undermanaged at the same time, cutting the budget for management is not exactly an effective response. One of the biggest problems in the NHS is a failure to coordinate care and that is a management and information problem that gets harder not easier when you have more medical staff to coordinate. And not just across organisations but inside them. In many hospitals settings the biggest failures in both quality and productivity come because the activity of different people is not well coordinated. This affects how we discharge patients in a timely way; it damages the throughput of operating theatres; it hurts patients because their medication is screwed up; it guarantees long waits in A&E because it is hard to find free beds (we don't coordinate the discharge process with the demand pattern for emergency beds).

So what?

The battle against statistical bullshit and Gresham's law must be fought. The more bad ideas are allowed to dominate debate and policy making, the less improvement will actually happen in the NHS.

Part of the problem would be addressed if the NHS collected better data about what actually happens on the shop floor. Too much of the data currently collected is focussed on top-down performance management rather than identifying and fixing operational problems. The central management style that demands ever more performance reporting (as criticised in this excellent rant by Nigel Edwards) drives out intelligent thinking about the root causes of problems (not least because it consumes so much of the scarce management time available for the operational managers who should be problem solving). Worse, the senior management of hospitals have a worrying tendency to collect data just for performance reporting while neglecting to collect the data they should acquire so they can understand the causes of their operational problems.

Even when we do collect useful information we tend to collect it slowly and make shamefully little use of it to derive operational insights. Patient-level data on A&E performance, for example, has had almost no influence on where money is directed in attempts to solve the persistent decline in A&E performance (see my argument here). We need to make more use of the big datasets for improvement and we need better tools to enable managers to get to those insights more quickly.

Having good analysis of the problem isn't enough. We also need to communicate those answers in ways that actually influence people. Data isn't convincing by itself: it needs to be turned into a message that works for the different audiences that can make a difference to what gets done. Partly this is about paying attention to how data is communicated: good data visualisation is an often neglected first step. But we also need to tell convincing stories. Bad ideas propagate not just because they are often unchallenged but because they are encapsulated in convincing, plausible stories. Gresham's law applies because the bad ideas sound more plausible than the good ones and attract more attention in the commentariat, the policy makers and the operational managers. Counteracting this with analysis isn't enough: we need better stories about what works as well.

The fight against Gresham's law must be fought. If it isn't, the NHS will continue to waste effort on initiatives that won't help it improve. Even if it eventually gets more money, much of that will be wasted because it won't be focussed on addressing the real bottlenecks to better performance. Britain's most loved public institution can't afford that.

No comments:

Post a Comment