The NHS is clueless about how to collect and use data

The NHS is deeply, fundamentally clueless when it comes to data. Data should be the single most important resource that helps the system meet the £20billion target for improving productivity and quality. But nobody, anywhere in a position of authority seems to have the slightest clue how the single biggest public dataset about healthcare could make a contribution to that challenge. And a whole pool-pah (it’s a Bokononist word, look it up) of problems flows from that cluelessness.

There is more than enough public argument about whether the controversial care.data programme is a good thing or not. The programme planned to join up GPs’ datasets about their patients with hospital datasets about the same people. The benefits to clinical research were supposed to be enormous (and probably are enormous). But controversy erupted because the issue of patient consent was treated casually and many campaigners don’t think patient data should be used for things like research without explicit consent from the patients.

But there is something missing from the debate that neither the advocates of care.data nor the privacy activists seem to have realised.  What is important is not the secondary uses of data (which means things like clinical research and drug development or other things which exploit collective data but don’t provide a benefit directly to the individual patient) but the primary uses of the data (which are relevant to the immediate care of the patient or the immediate operations of the NHS). And when it comes to primary use of the data the NHS is like a blind man in a dark room searching for a black cat that isn’t there. And, since informed consent is only valid if the patient understands the implications of their decision, most patients will not be making informed decisions whatever they decide.

Instead of talking about the primary uses of data the entire debate has focussed on sexy, shiny research uses (like a Magpie with an obsessive compulsion for shiny objects). The positive stories mix buzzwords like Big data, Genomics, Personalized Medicine, Predictive Analytics, Molecular Diagnostics, Graphene-enabled Neurophysiology Enhancers (there’s a story here with all of those buzz words except the one I completely made up). The public remain unconvinced, perhaps because these benefits seem like a long way away from their GP appointment next week or because the gains will mostly flow to rich private organisations like drug companies who the public already think make an obscene superfluity of profits.

But the worst effect of this focus on sexy long term secondary stuff is that the boring prosaic stuff is totally neglected despite its importance to the individual patient or the management who have to run the NHS tomorrow. And the NHS is not just ignoring the uses of the data, it is failing to think about how to collect and handle the data in ways that would make it easier and cheaper to collect and more useful to those who need to work with it.

Here is a strange observation about what the public thinks. When asked whether they are happy with their health records being shared with evil private sector capitalists (like insurance or drug firms) people are often a little wary. But when asked whether they think the NHS should share their GP records with, say, A&E departments they are not only happy about the idea, they assume it already happens. If an A&E department gives you a shot of penicillin and it kills you because of an allergy your GP knows about, that’s bad. And avoidable. And a good reason to share information across the NHS. And most people assume that sharing is routine. Most people are wrong. The majority of A&E departments could not access that essential information from your GP records at the point where it is most needed. In fact many hospitals would struggle to share that information internally among their staff even if you made a point of telling the admitting medic your medical history accurately when admitted (and many people are admitted in a confused state where their own recollection of their medical history or current prescriptions is not that accurate to start with beyond the knowledge that some of the pills they take are blue and others are orange).

The public, despite the fact that too many of them read the Daily Mail, are actually fairly sophisticated in judging the benefits of joined-up data versus the tradeoffs to their confidentiality. And they seem to accept the tradeoff.

The NHS doesn’t even seem to understand the problem. The people concerned with managing NHS data have not told the public any stories about these immediate and important benefits. Nor has the NHS collectively sought to manage its data to maximize the gains. This failure has a corrosive and debilitating effect on the quality of care and the NHS’s ability to improve that care.

Paul Baumnan, the current finance director of NHS England, has monthly reports of the financial state of the NHS. So a couple of months into the year he might know that there is a worrying financial overspend in hospitals. But he will be clueless as to why as the data about what activity is happening is not accessible for perhaps another 3 months and when he gets it it will be a lot less reliable than the data about the money.

Let’s imagine for a moment there is a short term public health catastrophe like an sudden outbreak of zombies in Manchester. We could nip the outbreak in the bud and prevent it spreading by issuing AA-12 automatic shotguns and plenty of ammunition to the admitting doctors in Manchester’s A&E departments. They could neutralise the outbreak by shooting the zombies in the head before they bit any of the other patients in the A&E queue and propagating the infection (i’m assuming fit people can run fast enough to avoid them). The NHS could order precautionary supplies of shotguns for all the A&Es surrounding the manchester area in case the immediate response didn’t work.

The real NHS would not notice the outbreak centrally for a good few months (or years if it had to wait for the publication of academic analysis in the British Journal of Zombie Medicine) by which times zombies would have organised themselves into a political party strong enough to stand for parliament and pass legislation to ban shotgun sales to doctors.

OK, I’m using the zombie example as a humorous aside but the NHS faces real problems where the same issue applies. Every winter, for example, it faces an A&E crisis. Waits in A&E extend to headline-making levels. And the system moves into crisis mode in a paroxysm of effort to try and address the problem. But it doesn’t know what the problem is. It can tell the problem exists as performance is reported weekly, but it can’t tell why. Is it because patients are sicker in the winter? The NHS doesn’t know because that data is only reported months after the performance data and isn’t available to most of the people who need it. The NHS should already be able to tell that the winter crisis isn't caused because more people are turning up (winter is quieter than summer) because that is reported with the performance. But the NHS lacks the analytical capacity or will to do that simple analysis and thereby avoid vast amounts of spending on solutions that won’t work. But there is no prospect at all of really getting a grip on the problem because the detailed data about why things are slower in the winter flows slowly and unreliably months after the crisis and can only be analysed by a select few and they are never operational managers inside the system who might be motivated to fix the problem  (like before the zombies get them or they get sacked because their performance is so poor).

Other organisations don’t deal with their critical information this way. For example, a savvy fashion retailer will know that the sexy red dress it launched last week is selling like AA-12 shotguns in zombie outbreak. As a result it can tell its manufacturer to triple the production run. Next week instead of angry scenes where sex-starved women fight each other in the isles for the dwindling remaining stock, it will make obscene profits from selling vast quantities of the freshly made dresses. It can do this because its key information systems are geared to recording the right information and its supply chain is geared to responding to the resulting analysis.

The NHS, despite being concerned with much more important things than fashion (like life and death), doesn't bother setting up its information collection or analysis to be able to do this. Doctors apparently have far more important things to do that working out whether the treatments they issued last week are killing people. Instead of being able to collect the right data, share the right data and analyse it so it does a better job of delivering high quality care next week than it did this week, it carries on with unreliable, paper based recording of the wrong, poor quality data and then doesn't bother to analyse it for any purpose at all for months if not years. Then it brings in new rules to minimise the possibility of any evil drug company using NHS data for commercial gain which has the side effect that nobody else (even the independent researchers whose motives are pure) can assess whether that company’s existing products are killing people or curing them. And the NHS can happily reassure patients that their data cannot be accidentally leaked in ways that breach their confidentiality (while forgetting to mention that it can’t be used to save their life or improve their care either).

OK, i’ve drifted into a rant. But there is a serious point here. The NHS doesn’t seem to have much of a clue about how patient data could be used to run tomorrow’s NHS more efficiently than today’s or how to make the quality of tomorrow’s care better than today’s. As a result it doesn’t organise the data it collects in ways that minimize the time or maximise the quality. And it makes the data hard to share for any use not just the distant secondary uses that worry patients.

There is a £20bn gap between the likely future budget for the NHS and the expected spending needed to keep the system functioning at current standards. The best way to bridge that gap is to use patient data routinely to improve the quality and efficiency of care.

This can’t happen while the NHS neither understands what those primary uses of data are nor invests in collecting or analysing the data.