A tale of two statistics and why accuracy isn’t about decimal places

A tale of two statistics and why accuracy isn’t about decimal places

I received an email late last night suggesting that I might like to blog on the sudden pulling of the property transaction figures – that is the recently introduced dataset that measures residential sales over £40,000.

I lead a thrill filled life…

Anyway the figures have plunged of late and the statisticians are a bit worried about accuracy of the data. They are rather in the limelight at the moment.

I replied to say I was considering making a comment but had to calm down. Pulling the numbers meant I had to recast various graphs I am putting together for the forthcoming Housing Market Intelligence report.

I admit to being prone to irritation – another of my less attractive traits, but I was sufficiently aware of my failing to avoid launching into an attack on the statisticians at HMRC.

I found alternative datasets for the graphs and I’m now a little less miffed, but I am still left with the same question that first came into my head, why are these numbers being questioned when the much more odd numbers for consumer spending have not been pulled?

That question was reinforced with the release today of another set of counterintuitive figures for consumer spending.

I have been a journalist for some while, so I guess my initial reaction was almost inevitably going to be one of suspicion. Queer “good” figures not pulled, queer “bad” figures pulled…um? Could there be a conspiracy to manipulate state data?

But for now I think I will leave that thought to one side. Because it sparked a thought about a broader issue and that is how we respond to numbers and statistics.

We are currently in a period of significant change, where we are seeing relatively large fluctuations (normally drops) in the numbers, with some in relative freefall. In many ways we are at a point where the accuracy of some data almost really doesn’t matter. It is probably enough to say things are bad, or very bad, if we want to ascribe any meaningful meaning. Frankly 0.1% either way on a 10% fall doesn’t really tell us anything.

But ironically each number seems to be scrutinised and picked over far more than ever, which I find a bit baffling. It seems that every new number that is released is being ascribed a meaning and appears to warrant debated. And I admit, to my embarrassment, that I get caught up in the excitement.

But I should know better. I once had an editor who drove me crazy with the meaning he would ascribe to numbers that in my view didn’t warrant that much attention. I recall in exasperation saying (loudly) “Just because it’s a number, it doesn’t make it right.”

In contrast I was fortunate enough to have a physics teacher (a cliché in many ways who spoke with a strange accent and played the violin) who would chastise pupils for putting unnecessary decimal places on to figures. He was damn right and I thank him for putting me straight on that – a number of times.

Putting extra decimal places on a number just because that is the way it comes out of the calculation doesn’t in itself make a figure any more accurate. If your measurement is accurate to one decimal place, putting three places on a figure is just a waste of ink and is actually distracting.

The point that I am labouring to make is that we place meaning on and ascribe accuracy to numbers that they most often don’t deserve.

For example, the stated figure for construction output in Great Britain in 2007 is £122,293,824,073.96. Not that any Government statistician would possibly seek to use that number precisely.

But anyone who thinks this is in anyway an accurate measure providing a “true” figure for how much construction (if we can even define the boundaries of such a thing) that was undertaken in 2007 should really think again.

If the stated figure is within 10% I’d be impressed, but that doesn’t make it a bad figure or the dataset a bad dataset. The more important question is whether quarter to quarter it is measuring broadly the same thing, because the reality is that it is very hard to measure construction with any degree of accuracy.

The trouble is that most of us take individual numbers too seriously without really trying to understand what they are describing or indeed what they might really mean. And in the excitement of it all, I fear I do the same.

Comments are closed.