Can we accept unquestioned the construction figures presented by the latest preliminary estimate of UK economic growth?

To do so is to accept that construction is enjoying workloads at a level that just three years ago were close to pushing the industry’s capacity to its limits.

What’s more the rate of growth being enjoyed by construction – 14% or so over six months – is seen by many who follow the numbers as astonishing and for some incredible.

But before launching into any criticism, the first thing to say is that measuring construction is extremely tricky and more so during times of volatility.

The second thing to say is that the industry has clearly enjoyed a spurt in activity generated in large part by the fiscal stimulus introduced by the previous Government which boosted both public and private sector work.

Despite that, questions still remain: Has the growth been as rapid as the numbers suggest? And is construction activity really back to near boom levers?

Few in construction would say yes to either let alone both, but as one sage of the figures said to me recently: “When you have excluded all the possible errors you can think of you are left with having to accept the figures.”

Perhaps he’s right.

But, there is very little other data to suggest that the construction industry is bouncing along at rates last seen in 2007 when talk of skills shortages filled the offices of industry policy makers.

Instinctively the figures don’t feel right and I am sure they don’t feel right to the 70,000 or so more construction folk sat on the dole this September than were sat on the dole three years ago.

And while the employment data – along with various other industry surveys – do suggest a healthy bounce back for the industry they do not support the notion that workloads are back to near peak levels.

So what could be wrong?

To try to get some feel for the likely source of any confusion with the figures we have to understand how the data are collect and processed.

And below I have tried to suggest some possible things within the data collection and manipulation processes that might be influencing the new figures or might have influenced the old figures in a way that might have resulted in the seemingly odd profile of construction activity portrayed by recent statistical releases.

The suggestions made are far from exhaustive, but hopefully illustrate some of the possible causes of weirdness that might thought to be lurking in the official statistics.

Many of the possible discrepancies mentioned below will have been examined over and again by the ONS along with plenty of other possible glitches.

The first thing to say is that the data series we are talking about is in fact two data series knitted together. At the start of this year the ONS introduced a new monthly construction output series which came up with its picture of construction activity. This series was then back cast to get a notional historic series.

For various reasons the old series data had to be constrained to fit with the total for all construction output, which was not as straightforward as it may seem and has presented its own set of problems.

So we should expect to find problems when we try to look at the two connected construction series over time.

Technically there are plusses with the new series. The sampling frame is on the face of it better than what was used before.

The old system used to exclude more firms and an estimate was then made using data gleaned from labour market statistics for the unrecorded output. This is now not done and also local authority direct labour organisations have been excluded from the survey (which could be seen as good or bad depending on your point of view).

(THING 1: have the changes in inclusions and exclusions resulted in a different profile of construction activity? Obviously, yes. But what is the difference and how does this vary during phases of growth and slump?)

There are concerns with the old series and there is suggestion that it may have been significantly undercounting the amount of repair, maintenance and improvement (RMI) work during the boom years. This was covered in an earlier blog and, in part, is considered in an Economic and Labour Market Review paper produced by ONS.

The possible amount of undercounting is probably unknowable, but it could potentially have increased peak construction by some billions of pounds. And it is not clear, if there was a significant undervaluing of the peak, how this might sit with the figures generated from the new survey.

(THING 2: was the previous peak significantly understated, so making the current level of activity further below the peak level than the data series suggests ?)

One notable structural change between the two series was that new work accounts for more of total output in the new series and RMI less.

RMI represented about 44% of all construction in 2009 under the old series, while under the new series it represents about 38%. This is a significant change.

Generally speaking, new work is more volatile than RMI, certainly on the way up out of recession. So if there is more new work in the mix we might reasonably expect to see more radical swings in output than was the case in the past, particularly on the upswing.

(THING 3: is the rapid growth partly a result of more new work within the mix?)

The methodology for dealing with outliers (numbers that are extreme in some way or another) has also changed. In the past the approach would be (I understand) to check up on odd results and, where no satisfactory answer was received, to exclude them – a trimming of the data.

The new approach uses a technique called Winsorisation, which in essence is an automated treatment of outliers that pulls them back into the fold, so to speak. There was also another technical change with turnover rather than employment being used as the auxiliary variable, which would be used to highlight potential outliers.

Without a detailed briefing I am cautious about drawing too many implications from these changes and, certainly, the ONS does use Winsorisation on its other business surveys, so in that sense it is being consistent.

However, I wonder whether the oddities of construction and its less homogeneous nature by comparison with other industries make this as appropriate. My experience is that “explainable” oddities are commonplace in construction.

The ONS is aware that this change will influence the figurers and has tested the effect. And I believe are continuing to test.

In an explanatory paper on the change in the output figures it said: “The results showed that the overall estimate of output, less the value of unrecorded output and other post-system adjustments, increased between 1.3% and 2.1%, simply by this change in auxiliary.”

It added: “The combined effect of the sampling frame and design changes, together with the impact of Winsorisation, increased the output estimates, net of the unrecorded output value and other post-system adjustments, by between 3.0% and 5.7%.”

My gut feel is that this may be an important factor and, instinctively, not knowing the precise way this process works on the data, one would suspect that its bias would be to expand the output in comparison with the old series, which indeed is what the tests seem to show.

If the bias is consistent and even this may not be a pertinent issue with regards to the apparent problems, but this may not be the case.

(THING 4: could the new methodological approach be boosting growth rates?)

A further important point to understand about this survey – and indeed any survey – is what is actually being measured. This survey seeks from respondents a “value of work carried out” over the given period (month). The contractor then has to exclude various associated payments to the likes of subcontractors.

Measuring that accurately is a pain, so the likelihood is that firms use the value of invoicing, because that would be the most straightforward set of data at hand to the poor person conscripted to fill in the survey return.

But we don’t know exactly. They may use payments received. They may genuinely try to give an accurate measure of work exactly done, if they have the management information systems that would allow it. And how they match this the related payments to subcontractors etc is a further issue.

This clearly introduces the potential for various and variable lags, given that both measures from invoices or cash in are generally after the event.

(THING 5: are we seeing in the figures for a given month data for work actually carried out a month or two months earlier?)

The lags effects this might introduce would be more smoothed on a quarterly basis but might become misleading within monthly figures.

Perhaps more worrying for people like me is that I haven’t heard a satisfactory answer as to exactly how firms are filling their survey forms.

If invoices are being used, in construction there can easily be a significant difference between what is invoiced and what is eventually paid. This is not too much of a problem if the difference is consistent. But I suspect that it is far greater when we are in recession, which would lead to a higher figure for the current price measure of output than is the case during periods of more normal trading.

(THING 6: could a greater underpayment of invoices be leading to a relative overestimation of output during periods of recession?)

The new series seeks to measures output monthly rather than quarterly. So firms are filling in forms more regularly which should be better as respondents will be more likely to set up systems that are more consistent month to month than they would have been quarter to quarter.

But given that invoices are often sent out for a second or even third time, with monthly recording there is a possible increase in the risk of double counting of work done (and also, probably to a lesser degree, of work not being recorded), with firms forgetting to exclude repeat invoices that would have been picked up when compiling data on a quarterly basis.

(THING 7: is there more double counting within the survey returns than in the past?)

When the ONS has captured its data it then has to gross up the sample to get a picture of the whole universe of GB construction. For this I understand they quite rightly stratify the sample by firm size.

But what we know about construction firms’ behaviour in a recession is that the big players start to bid for much smaller jobs than they would normally. This boosts their work and shrinks that of the smaller players.

I don’t know the process by which the data for each stratum of the survey is grossed up to get to the final output figure. But if the system is not sufficiently accommodating to the effect of larger firms drifting down the food chain we might expect an exaggerated final result through underestimating the loss of work/firms/trading in the lower strata – particularly given that turnover among smaller firms can be very volatile.

Also any relationship used to scale up from the sample of smaller firms will be based on historic data which, in a recession, will probably lead to an overestimation of the amount of work produced by smaller firms en masse.

Opening the question out more broadly we might ask whether we are seeing some effects of survivor bias.

(THING 8: is the work carried out by smaller firms being overstated as a result of the squeeze on small firms during a recession?)

One area that has received plenty of queries is that of the deflators used to convert cash figures into estimates of output in volume terms. Producing accurate deflators is hugely problematic in construction compared with many other industries because of the extreme heterogeneity. Trying to find like for likes to compare is not easy, so measuring inflation or deflation is not easy.

But much work has been done in discussion with the Department for Business, Innovation and Skills, the Building Cost Information Service and the surveyors’ body RICS to improve the deflators used. But doubts naturally persist, because faulty deflators might explain some of the unexpectedly strong growth.

If, for instance, there is in fact some inflation in construction prices as a result of a bounce back from previous sharp drops, and the deflators – as they do – suggest prices are still falling, then this would mean the volume of work estimated would be increased rather than decreased relative to the current price of cash estimate of workload. The difference would not necessarily be that great, but if the deflators were later revised, say, to accommodate inflation rather than deflation it would lead to shading down of the growth estimate for construction.

(THING 9: are the deflators accurately reflecting real price changes?)

This then leads to a very important point – perception. Because prices and margins are still uncomfortable and because most folk in construction are (quite rightly) expecting a second wave of the recession when the cuts in public spending feed through, there is, understandably, going to be a view that things on the ground can’t be as good as the figures suggest.

Furthermore much of the growth has seemingly been in the civils sector which tends to employ fewer people and so would be felt less by those that have lost their jobs.

(THING 10: are the figures more representative of the real world than we think?).

It is hard to know exactly what might be at fault with the construction output data series. There will certainly be faults, the question is how large.

Certainly, in my view, it is not hard to imagine that the previous series underestimated the peak and that the current series is overstating growth, certainly relative to the old series. Most of the suspicions one might have concerning the data and the methodology does seem to suggest an outcome of overstating rather than understating growth in the figures at this stage of the economic cycle.

If this were the case then it is easy to see how we have ended up with figures that – although not dramatically different in scale to what we might expect – appear extremely unrealistic when looked at as a series.