Evaluation of policy: the task of quantifying the unknowns: deterrence and non-detection

From the Autumn 2012 Research Bulletin, by Stephen Davies, Professor of Economics and Peter Ormosi, Lecturer in Competition Policy

For all the considerable recent advances in techniques used to evaluate the impact of competition policy, we still know remarkably little about two major unknowns: how much anti-competitive harm is avoided because it is deterred, and how much harm is there out there of which we are blissfully unaware, because it is undetected? This article discusses some of the very real problems, revolving around selection bias, in our own current research designed to quantify these unknowns.

As economists, we are brought up, almost from the cradle, to believe that ‘competition is a good thing’, and it follows that, if effective, competition policy is a worthwhile component of any government’s expenditure.  In recent years, governments and academics around the world have devoted considerable effort to estimating the impact of competition policy on the economy and on welfare, and their findings are generally very positive.  For example, in the UK, the Office of Fair Trading calculates that every £1 it spends yields £10 of benefits to consumers.  Evaluations by the European Union and the USA yield qualitatively similar results.

However, all such evaluations are based only on the cases that a competition authority (CA) prosecutes/intervenes in – cartels which it busts, anti-competitive mergers which it blocks or remedies, etc.  We call these the ‘observed’ cases.  This raises an obvious question ‘what about the unobserved cases?’  For present purposes, we highlight two reasons why a case is unobserved: deterrence or non-detection.  On the one hand, good law and enforcement deters antisocial behaviour: this is hopefully its most important function, and the magnitude of deterred harm should be, but never is, included in impact evaluations.  On the other hand, if a CA fails to rectify anti-competitive harm because it does not detect it, this is a foregone opportunity, or even a failure of policy.  In other words, we should ask not only what are the direct observable benefits from competition enforcement, but also (i) the indirect benefits because the Law and/or the CA deters firms from behaving anticompetitively, and (ii) the costs to society from that anticompetitive behaviour which goes undetected.

Research which attempts to quantify these two magnitudes is still in its infancy, but we do have some estimates. In a rare qualitative study commissioned by the OFT, interviews were conducted with lawyers, economists and companies involved in competition cases. From the survey of legal advisers, the research suggests that, for each merger blocked or modified by the CA, there were at least another 5 proposed mergers, that were abandoned or modified on competition grounds. This ‘deterrence multiplier’ was even higher for commercial agreements and cartels.  Elsewhere in the literature, various economists have attempted to estimate the ‘detection rate’ for cartels – a typical finding is that about only 1 in 7 cartels is actually detected.

One might argue with the methods and data employed in such research, and the present authors do have severe reservations.  However, even if these ‘headcount’ multipliers are accepted at face value, there remains a potentially much bigger issue – the possibility of sample selection bias.  To see how this works, assume hypothetically that, for every cartel detected, there are another 5 which are deterred and a further 5 which go undetected, can we simply assume that the magnitudes of undetected and deterred cartel harms are each 6 times greater than estimates of savings achieved from detected cases?  The answer is no, because the observed sample (undeterred but detected cases) may not be a random sample of the full population of potential cases (i.e. observed and unobserved.) 

So, can we improve things by quantifying the magnitudes of harm, rather than estimating the number of cases?  In fact, this is daunting task because, by definition, we have no information on cases that are unobserved – because they are deterred and do not occur, or because they occur but we don’t know of their existence.    

To give a flavour of our research, consider the deterrence of anticompetitive mergers. First ask the question: ‘what is likely to be the most profitable merger for the firms, but most harmful for consumers?  Ceteris paribus, theory tells us that these would be mergers where 2 duopolists merge to become a single monopolist.  Next, ask ‘how many such mergers do we actually observe in the real world?’  The answer is very few, and almost certainly the reason is that such mergers are rarely proposed because the firms know that they would be blocked by a CA.  In other words, deterred mergers may well be far more harmful than the cases actually investigated by competition authorities.

Now consider cartels. Most theory on cartel formation assumes that potential members base their decision on whether or not to cartelise on the expected profitability of the cartel, bearing in mind the probability that it might be detected and penalised.  If so, it is likely that it will be the least profitable potential cartels which will be deterred.  On the other hand, and admittedly more arguably, it is the most profitable cartels which are not only undeterred but also undetected.  The latter would be true, for example, if the temptation to apply for leniency was greatest for members of cartels which are least stable and least profitable.

In terms of ‘harm’, these two examples suggest that (i) the gains from deterrence may be even higher than currently thought for mergers but lower for cartels, and (ii) undetected harm from cartels is even greater than is currently thought. Our ongoing research employs a mixture of economic theory and careful econometric ‘controlled experiments to explore these issues more rigorously. 

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: