Forget anecdotes. If you want to know what’s going on in the real world, look at a spreadsheet

One of the occupational hazards of taking data and statistics seriously, and using social media to do so, is frequent accusations that my focus on hard numbers means that I have my head in the clouds, or buried in a computer, and am therefore somehow detached from “reality”.

I’ve lost count of the number of times a Twitter link to a chart, research paper or Office for National Statistics data release, making a point about what the evidence shows, is greeted by some version of “why don’t you stop looking at spreadsheets and get out into the real world?”

This is particularly the case for immigration. The response to the overwhelming evidence that, in the UK, there is no measurable impact of immigration on employment, and only a very modest impact on the wages of the low-paid, is often an anecdote, a reference to (perceived) personal experience, or just an injunction that I should spend more time in the pub.

Here’s a recent typical example (amazingly and embarrassingly, from Dr Niall McCrae, a lecturer at King’s, my own university, in a scientific discipline):

"Jonathan Portes gave an accomplished performance, asserting that immigration has boosted society, and that it hasn’t depressed wages. Such thinking may be lapped up by a forum of intelligentsia, but try telling that to my landscape gardener friend, who has been severely undercut by waves of east Europeans."

I was reminded of this by a tweet, attached to a passage in David Goodhart’s new book:

"In recent years the falling relative pay for basic jobs, the overwhelming stress on mobility and educational stress, the hourglass labour market and the apartheid system created by a mass higher education system have all made it harder for the mainly Somewhere people doing routine jobs to feel valued and dignified in the modern economy."

This, Daniel Bentley at Civitas argued, is a “really underappreciated point”.

Underappreciated, perhaps, because it’s wrong.

Leaving aside the truly bizarre apartheid analogy (Mr Goodhart has a habit of this sort of thing, as I've noted before) the claim that “in recent years” relative pay has fallen for basic jobs is simply false.

It’s true enough that earnings inequality did widen sharply in the 1980s and early 1990s. But this isn’t exactly “recently”, and has absolutely nothing to do with the post-1997 immigration Mr Goodhart blames for much of the UK’s contemporary problems.

Indeed, over the past decade, if anything, boosted by increases in the National Minimum Wage, it has improved somewhat, as this chart shows:​

Now I would argue that this point, that workers at the lower end of the earnings distribution have done particularly badly, is pretty fundamental to the entire argument Mr Goodhart is making (indeed, the importance of this paragraph to Mr Goodhart’s thesis is presumably why Mr Bentley highlighted it).

So you might think that getting it wrong matters rather a lot. But I have no doubt what Mr Goodhart’s response will be.

As with my review of his earlier book, The British Dream, which listed (very much non-exhaustively) a number of his more glaring factual errors, he would say that I am “sniping in the footnotes” and that I spend too much time with databases and not enough in the “real world”.

But there is a fundamental problem with the argument by those like Dr McCrae or Mr Goodhart that “spreadsheets” or “databases” are somehow divorced from reality, while the experiences of (selected) individuals represents it.

In fact, spreadsheets – or at least the ones used by labour market economists and, indeed, quantitative social scientists more broadly – are far more closely connected to the “real world” than any individuals’ experience can hope to be.

Consider the Labour Force Survey (LFS), the primary data source for economists analysing the UK labour market. Each quarter the LFS samples 40,000 households, covering 100,000 individuals, a representative sample of (broadly) the UK resident population; lengthy interviews are conducted in person (and subsequently by phone) and cover a wide range of topics in considerable detail, from education, earnings and employment to age, marital and family status, country of birth, and disability.

So when I say that the evidence is clear that immigration doesn’t impact on the employment of UK-born residents, this analysis is formulated in terms of numbers on a spreadsheet or data points in a regression. But behind those numbers are what tens of thousands of real people have told professional interviewers, and in a way which means that the results are in turn representative of lived experience of the UK population as a whole.

So the statement that, say, the bottom decile of full-time workers have recently seen their pay rise faster than average is not (just) a statement about numbers, or a claim that Mr Goodhart has failed to read the right ONS spreadsheet.

It is a statement about what has, contrary to Mr Goodhart’s claim, actually happened to the pay packets of several million people.

It is the spreadsheet, not what my nephew looking for a job, your cab driver, or Dr McCrae’s landscape gardener friend say that best reflect the real world.

A similar, but even more toxic, disjunction from reality is seen in those who claim that poverty is not about money.

For example, Fraser Nelson, the editor of The Spectator, frequently claims that the Labour government saw poverty solely through the lens of numbers, and that the Brown strategy of attacking child poverty by redistributing money to the poor via tax credits was simply manipulating numbers on aspreadsheet:

"Someone who is nudged just above this threshold, with an extra £10 a week, is deemed to be 'lifted out of poverty' although the people concerned would be astounded to hear themselves so described. If they had a family, then their children would be described as being 'lifted out of poverty'. So, by precision-bombing the right people with tax credits, you could claim to have lifted hundreds of thousands of children out of poverty.... instead of fighting poverty, the Labour government spent billions manipulating a spreadsheet – to catastrophic effect."

[UPDATE: Fraser Nelson doesn't like this description of his views. I discuss his contradictions here]

And indeed it is true that this approach was shaped and driven by numbers on spreadsheets – numbers which represented hundreds of thousands of low-income families with more money in their bank accounts to spend on food, shoes or the occasional holiday.

There’s plenty of evidence that’s exactly what happened, with commensurate improvements in child welfare. A comprehensive review by the Joseph Rowntree Foundation concluded – unsurprisingly to anyone who knows anything at all about the topic: “Children in low-income households do less well than their better-off peers on many outcomes in life, such as education or health, simply because they are poorer.”

In other words, despite Mr Nelson’s rather convenient assumption, the spreadsheet-driven policy – by ensuring that low-income parents had enough money to look after their children – worked in the real world.

None of this means that talking to people, or, more relevantly for these topics, rigorous qualitative research, which is generally not what the critics mean, is not useful and sometimes necessary to get a full picture. But suggesting that data doesn’t represent reality as well as personal experience is simply the opposite of the truth.

If you want to know what’s actually going on in the real world, look at the data.

Jonathan Portes is professor of economics and public policy at King's College London, and senior fellow in UK in a Changing Europe. This article originally appeared on his personal 'Not The Treasury View' blog