I’m surprised to see the aphorism that competition in the NHS saves lives still being trotted out as if it is some kind of truism – most recently by John Rentoul this weekend. So let’s be clear what the LSE research actually shows us.
For the claims around competition saving lives, in my reading of the work, the following line of argument has to be true.
- Choice of provider was introduced by Labour in 2006.
- Therefore competition was introduced at the same date.
- Take a measure of competition which does not correlate to how urban an area is.
- Take a measure of mortality in an area which is not, itself subject to choice (AMI – heart attacks)
- Look to see if the measure of competition which does not correlate to how urban an area is correlates to the measure or mortality in an area, which is not, in itself, subject to choice (still with me?)
- If yes, with a range of econometric controls, say competition saves lives.
Now given all this, do you think these claims are credible?
First does choice=competition, when only half of people surveyed by various bodies say they were even offered it, and only a tiny proportion actually used any kind of information sources when making it (have a look at http://www.lancet.com/journals/lancet/article/PIIS0140-6736(11)61553-5/fulltext). In other words, even though only half of people being offered a choice remember it, and only a tiny proportion could actually have made an informed choice, that counts as competition.
Second, the measure of competition used in the study does not correlate to how urban an area is. In other words, competition is not restricted to cities (as we might intuitively expect, given that’s where most providers are), but is a measure wider than that. It’s a measure of competition which doesn’t seem to be related to how dense provision is. Does that really make any kind of sense to you?
Third, the measure of mortality is not itself, based on an area of clinical work where patients can make a choice. The authors have done this on purpose – their claim is that the effects of choice (and so competition) are so profound that they even affect areas where no choice is itself in place. Now that’s a pretty extraordinary claim – for 60 years we’ve been trying to reform the NHS, but simply introducing choice (which, remember, patients can’t remember being offered and only a tiny proportion made in any informed way) led to clinical areas which aren’t even being affected by choice improving! That’s pretty extraordinary. You’d expect some explanation of how this happened being given, but it seems, it’s simply down to the magic of choice.
Fourth, you run regressions of your measure of competition (which isn’t about how urban an area is) against your measure of mortality (which isn’t affected by choice), and you get a positive result.
Finally, you proclaim that competition saves lives. You are listened to because people don’t understand your paper as it is mostly made up of mathematical symbols, and uses methods from quantitative economics. And economists have SUCH a good record of conducting great research in their own area (financial crisis anyone?) that they are now applying their methods to areas like healthcare as well.
You may have picked up from the above that I’m rather cross about this. We need informed and careful debate about the future of the NHS. This kind of work really, really, doesn’t help.
Ian Greener's Academic Blog 
February 13, 2012 at 11:09 am |
I think it’s a little extreme to suggest that because of the financial crisis, all work done by economists, and all their methods, are discredited. Now I am pretty sure you don’t mean that, because you cite Albert Hirschman in another recent post. But it’s a bit of a straw man.
Anyway, on the substantive points (from my memory of a seminar discussion on this at LSE a few months ago):
The reason the researchers choose a measure not directly subject to choice is to isolate two different effects.
The first effect of choice is simply a selection effect. Assume that there is some diversity in provider quality (not a controversial assumption I think). Then, where patients have a choice of service provider, well-informed patients might simply all move to the one with better results, leaving the old one to struggle with its higher mortality rates. This would mean that some patients (typically middle class ones) would get an improvement in services just by moving to a provider which happens to have higher quality already, while less mobile or worse-informed patients would be stuck with a provider that happens to be of lower quality.
The researchers wanted to test whether competition would actually improve outcomes even for patients who don’t or can’t move, and that’s why they use AMI mortality. They find that it does. The mechanisms by which this happens are not directly measured in the research, but the researchers have some reasonable hypotheses: for example an improvement in quality in the services that are subject to choice could have a spillover effect to those that are not. In addition, the very existence of competition in most markets, and the possibility of customers (and by extension patients) switching to other providers in the long term, gives organisations an additional incentive to improve.
I don’t quite understand your point about the measure of competition not being correlated to population density. Do you mean that the researchers did not control for population density in their measure of competition? I do know that this was an important issue for the researchers in how they designed their measures – because they wanted to eliminate the potential confounding effect of urban density as an independent cause of improvement in outcomes – but I am not quite sure what you mean here.
I like your blog, and am glad these important points are being discussed, and I certainly accept that the research base is not yet sufficiently deep or widespread to be compelling. But I do think the researchers have attempted in their work to address some of these potential challenges.
February 13, 2012 at 1:21 pm |
Hi Leigh, thanks for your comments.
The first point I’d raise is that this research is making extremely strong claims, but for me, don’t have the evidence to back them up. When you are claiming that competition saves lives at a time when the NHS is undergoing reform you have a responsibility to behave in a measured and careful way – and I don’t think this paper does that.
Second, the issue of choice and competition and urbanity are all tied up together – I don’t believe introducing choice had any effect on care – the piece I cite in the Lancet, which I helped co-wrote, I think drove a truck through the very simplified model which has been presented in this work, and gave more reasonable explanations for what they purport to have found. How can competition not be related to urbanity – urban areas have more providers – but the variable chosen by the researchers seems to try and avoid competition being about the number of providers that people might reasonably take into account when choosing (the small number that actually do). In short, the variable measuring competition doesn’t do the job. And that isn’t helped by the data excluding private providers, which is completely bizarre when making claims like this.
Third, I really like good economics – what I object to is the rubbish that has come to dominate the discipline based on over-simplified models and high maths. Economics has lost its way. That kind of work did contribute to the financial crisis – it gave a bunch of ideas around markets clearing and stock markets being efficient credibility when they should not have had it.
Finally, I don’t find the mechanisms suggested by the researchers plausible. What I object to most is that they didn’t bother to go actually ask the providers – they just assumed that was how they were behaving. They inferred it from their data – and there are better conclusions that could be drawn.
In short, this research has given legitimacy to a set of government reforms when it doesn’t have the evidence, in my view, to back it up.