Knee-jerk ideology makes any sensible debate on the NHS virtually impossible

The NHS needs to improve. Most people agree with that. But when it comes to what specific policies or actions will deliver improvement there is far less agreement. So sensible debate about what to do would be useful. Sadly we are unlikely to ever have that debate when the response to any suggestion consists of a storm of ideological name calling.

A recent opinion piece on the BBC's Newsnight programme about the NHS has led to a storm of protest and abuse. The piece consisted of a short video by Kate Andrews of the IEA (a free market think tank whose funding is not very transparent). The response mostly consisted of arguments like this:

1. The BBC should not provide a vehicle for right wing propaganda
2. Kate Andrews doesn't understand the NHS because she is an American
3. The IEA is an evil propagandist for an insurance based system and NHS privatisation
4. The USA's health system is evil
5. The BBC is biased

And more on the same lines.

Before I watched the video I assumed that she must have argued that the NHS needed to be financed by user charges and broken up into an american style mess run by the private sector for profit. Then I watched it. And my reaction was "what the holy fuck are those commentators talking about?"

The key arguments in the video are this:

1. The NHS is in a state of perpetual crisis
2. There is little appetite for radical reform
3. Campaigners often pretend that there are only two alternatives: the NHS or the US "system"
4. The NHS isn't on reasonable metrics "the envy of the world": outcomes are better in many other health systems
5. The USA is a crap system but is also an extreme outlier and a meaningless comparison
6. Many other countries (Australia, Singapore, The Netherlands, Germany, Sweden…) have universal healthcare with better outcomes than the UK
7. Competition between a variety of providers (both profit and non-profit) is a common factor in other universal care systems
8. Market reform in the NHS would be good

And that is it. No calls for abolishing "free at the point of use". No calls for privatisation. No praise at all for the US system.

The only contentious call is for more market reform in the NHS. The most left-wing pro-NHS campaigners agree with point 1. Point 2 is arguable but not controversial (unless you think that abolishing the Lansley act is radical which it isn't as returning the NHS to previous legal and structural model is a big change, but a conservative one). Point 3 is simply a summary of what the majority of commentators and campaigners seem to argue. Point 4 is solidly based on facts. Point 5 is also uncontroversial (though the fact that someone from the IEA has just said it ought to be interesting and pro-NHS campaigners might like to quote it a few more times in their arguments). 6 is also true but, annoyingly, rarely mentioned in the debate about what we should do to improve the NHS. Also 7, which probably explains why left-wing campaigners rarely mention 6 as it undermines the simplistic idea that provider competition is the root cause of everything wrong with the NHS. 8 is admittedly contentious and worth arguing about, though not for the reasons campaigners would normally use.

On that last point it is worth a quick diversion to see what good arguments against it would look like. The best argument isn't that competition doesn't work: it does, even in the NHS as respected and non-ideological health economists like Carol Propper have shown. But those studies showed that the benefits to quality were, likely, small (though the NHS's experiments with competition for elective procedures were not very radical). More importantly, more competition would do little to address the immediate short-term problems in the NHS even if they improved quality in the long term. We have more important things we need to tackle right now like gross underfunding, especially of capital and improvement projects.

So I can take Kate Andrew's argument and deal with it like an adult, specifically looking at what she argues for and rebutting it with logic. A disturbing amount of comment, however, looks less like a fight in a primary school playground and more like a shit throwing contest between two opposing tribes of agitated chimpanzees (seriously, read the twitter thread on the Newsnight post).

It is hard to judge whether any of the shit-throwing commentators even watched the video. I thought more might have latched onto her criticism of the US model ("even the IEA think healthcare in the USA is rubbish!"). What we actually got as a typical response was a series of ad-hominem attacks on Andrews and the IEA coupled with damning criticism of BBC bias for daring to allow such dangerous views to be broadcast (despite them clearly being labelled as an opinion piece to provoke debate and comment). Yes, the opaque funding of the IEA should make us suspicious of what they say and forensic in examining the facts and arguments they make, but it doesn't automatically render what they say as mendacious nonsense.

Much of the comment validates a point I have made before: it is impossible to discuss the NHS in ways that might help improve it because the "debate" is is conducted in clichéd shibboleths where all that matters is proving which side you are on. Identifying real problems and fixing them is utterly irrelevant.

If we really want to fix the NHS (and spend the government's new found largesse well) we need to dispassionately analyse what the real problems are and apply the new money to solving them. We need to look objectively at the facts and not just ignore the ones that don't entirely match whichever ideological agenda we have. The intemperate shit-throwing of the pro-NHS commentariat does not exactly encourage that debate. And the likelihood of finding good ways to make the NHS better is substantially lower as a result.

Why big, centrally-driven, IT projects fail (especially in the NHS)

Big IT systems developed top-down fail for the same reasons that centrally planned economies fail. Central planners have few clues about how to improve the work of the doctors and nurses on the shop floor. Better to let people solve their problems bottom up.

[Note:This article was originally written just after the coalition government announced it was dismantling the giant programme the Blair Government started in an attempt to speed up the computerisation of the NHS, the National Programme for IT (NPfIT). But the lessons are still true.]

So, finally, the government is facing pressure to put a stake through the heart of the Flagship NHS IT project. Uncle Tom Cobbley et. al. are wading into the debate with what they think should have been learned. Most of these explanations will be wrong, some might contain partial, biased versions of the truth, none will be useful in preventing the next big epic fail.

The real lesson is nothing to do with IT, not about who should have been consulted, doesn’t relate to who was the responsible officer and won’t be fixed by more ruthless contracts or better project management, thought these will all be suggested as solutions. The real lesson is one that government needs to learn for many of its big projects, not just its big IT projects.

The real lesson is also nothing to do with the project having the wrong goals. Most of what it was intended to achieve is highly laudable and would benefit the NHS and the health of the nation if it was achieved. It is hard for many outsiders, for example, to imagine how the NHS functions at all without shared accessible computerized clinical records. Providing effective patient management and clinical record systems for hospitals can’t be bad.

But how should this be achieved? The Blair Government was persuaded that the most effective way to do it was to plan it from the centre. This would enable economies of scale, guarantee system compatibility across the country and enable the buying power of Whitehall to cut better deals with big powerful suppliers who might be expected to bully weak hospital management into wasting money. And these benefits were mostly achieved. Several big firms withdrew such was the ruthless pressure on performance and cost.

The problem was that economies, cost effectiveness and compatibility are a lot less important than systems that actually work for the people who use them. And this problem is multiplied many-fold when the users are as diverse as the NHS. Even within a single hospital it is hard to satisfy all the departments with a single approach (example: users in A&E need to do many small things quickly, a system with a 30s delay for login and user authentication is essentially useless to them, yet many systems are designed that way as most users in the rest of the hospital don’t mind). Yet the centrally driven plan essentially tried to satisfy everyone with a system designed centrally. But nobody in the centre can ever get this right especially when many of the real needs only manifest when users start using the system. And this isn’t the sort of problem that can be solved by more extensive consultation with the users: they may not have a clue what they need and may not find out unless they are fully engaged in testing. Many of the best solutions might not emerge until there has been a large amount of experimentation but that is anathema to a centrally driven project.

Big enterprises driven from the centre have always suffered from these failures. It is the same reason that centrally planned economies are the ultimate epic fail in economic history: whatever the benefits in theory of a system without the messiness, costs and inefficiencies of a pluralist market economy, they don’t work in practice. When central plans fail, they fail for the whole economy (or the whole NHS) and they take longer to fix as the feedback that things are not working is suppressed throughout the system as nobody is rewarded for admitting the system doesn’t work. It isn’t that the players in pluralist economies don’t make mistakes: any given business is just as likely to screw up as any government planner. But, in a pluralist economy, there are many businesses so one failure doesn’t screw the whole system. And businesses are disciplined by their customers: if what they make doesn’t sell there is no escaping the failure, so they spot them and correct them faster. Because there are many experiments, there is more information about what works and what doesn’t work, innovation that works is rewarded and information about failures spreads rapidly. The gains in rapid improvement vastly outweigh the inefficiencies that come from the smaller scale and the coordination problems with multiple parties. Disciplined pluralism thrashes centrally driven planning every time by a large margin.

It might sound like this is some abstract economic bullshit and can’t apply to big NHS systems that need to be coordinated and need to have minimal standards of quality for the good of patients. But it isn’t. And there is even an example where a decentralised approach worked. Better still it generated one of the best and most effective IT systems in healthcare and it is a system which delivers far greater benefits than anything the NPfIT has ever hoped for. The system was developed by the US Veterans Administration (responsible for the hospital care of military veterans in the USA). The system was built as a series of modules, skunk-works style and in the face of strong and sustained opposition by the IT leaders in the VA. But groups of IT savvy doctors knew they could make big improvements in their daily clinical work if they had systems that did a better job with patient information (one part was developed to prevent patients being given the wrong drugs, a serious problem for them at the time). The skunk works used open source techniques and designed in the ability of each small module to talk to other modules. Eventually they got a modular system that allows any VA doctor anywhere (in any GP office or any hospital) to access all the relevant information about one of their patients. The clinical benefits were instrumental in raising standards across the VA from some of the worst to, by some experts’ reckoning, the best in the US health system.

So disciplined pluralism works even in IT projects. Better still, the more complex the need, the better the benefit of this approach will be. Governments are unlikely to adopt it though, as they can’t admit how little they know about what the NHS needs or how to deliver it. And most lobbyists will just encourage this belief by claiming that we just need to consult more doctors or hire better project managers.