The OBR: what would Einstein say?

Among the many utterly brilliant things Albert Einstein is known for is one of my favourite one-liners:

"Insanity is doing the same thing over and over again and expecting different results"

Apparently, it is now accepted that this "law" of nature may not apply in the quantum world, but for everything larger than sub-atomic particles, it appears to be so, and I, for one, will not be picking a fight with the great man. 

I ask this hypothetical question because I am constantly perplexed by a collective willingness in the world of economics for important institutions to pay heed to forecasts from organisations whose forecasting track record would indicate that there was something inherently wrong with their models. Would those important institutions not be guilty of a version of "Einstein insanity" by basing their economic behaviour on the forecasts of those organisations?  And is it not incumbent on the recipients of these forecasts to investigate the reasons for these repeated errors? Failure to do so would surely condemn them to repeat the mistakes of the past.

To be more precise, I would like to know why the UK Government places so much reliance on economic projections provided by the Office for Budget Responsibility, whose UK GDP forecasts vs outcomes since its inception have been as follows. 

The OBR was created in 2010 to provide independent and authoritative analysis of the UK public finances. It publishes forecasts twice a year to coincide with the Government's major fiscal events, namely the Budget in March and the Autumn Statement, and it uses these forecasts to assess the Government's progress against its fiscal targets. Of course, in doing this, crucial elements of the economy's performance will dictate how well any government is performing against its targets because changes in how much the Government spends and receives in tax depends to a significant extent on how fast the economy grows. But the OBR's projections are not just about assessing a government's performance against a plan; the government's policies themselves are framed by what the OBR analysis says, and of course, the media's assessment of those policies is very much dictated by what the OBR says.

Clearly, the OBR can be forgiven for not anticipating the arrival of COVID-19 in February 2020. Still, as you can see, over the 14 years since the OBR's inception in 2010, its forecasting record could hardly be described as accurate. Over the entire period, in only two years was the OBR within 0.1% of the GDP outcome, and in three years it was 1.0% or more too pessimistic. To put this into context, 1% of UK GDP is currently £22.7 billion of output. Remarkably, in 2021, the OBR's forecast was 4.6% too low, which amounted, in that year, to about £100 BILLION of output. Perhaps most perplexing is the post-COVID period during which the OBR has been consistently too pessimistic in anticipating both the economy's recovery from that event and its ability to cope with the energy price crisis that followed it. The OBR has made the same mistake again in being too pessimistic about growth in 2024. Last week's Q1 GDP data was much stronger than the OBR had expected, and it will now need, as a consequence, to upgrade its growth forecasts for the year as a whole.

Looking at the data in more detail, it is interesting to see that, excluding 2020 for obvious reasons, there is a consistent bias in the data in that the cumulative underestimates of growth are about 10x the cumulative overestimates. I am not a statistician, but that seems to indicate some structural bias or systematic error in how the OBR's UK economic model works.

The OBR is a very powerful organisation whose analysis affects us all. With that great power should come great responsibility. Unhelpfully in this context is the fact that the OBR essentially marks its own homework in the form of its annual Forecast evaluation report, the most recent of which was published in October last year. In 2023, it also published a working paper which looked at the OBR's forecasting record and compared it to other forecasting bodies, including the Bank of England. Interestingly, the report identifies what the OBR describes as "a tendency to overestimate real GDP growth and underestimate government borrowing". Quite how that statement in relation to growth is compatible with the forecasting record shown above is not at all clear to me.

Maybe the OBR, rather than relying on its own analysis of its performance, should also do what the Court of the Bank of England did last July and commission an independent review of its forecasting approach (ex-FED governor Ben Bernanke conducted the Court's review). Bernanke's report, which was published last month, was very critical of the Bank's forecasting process and made several important recommendations, including: "Over the longer term, the Bank should undertake a thorough review and updating of its forecasting framework, including replacing or, at a minimum, thoroughly revamping COMPASS” (COMPASS is the Bank's baseline economic model).

I thought this quote from the report was of particular importance in relation to the points I am making about the OBR.

"The goal of the forecasting process, of course, is to help the MPC make better policy decisions and to effectively communicate those decisions to the public." 

Maybe the time has come for the OBR to take a leaf out of the Bank of England's book, employ a little institutional humility, and ask Ben to take a proper, independent look at its forecasting framework with the ultimate aim of helping future UK governments make better informed fiscal policy decisions.

I am also not aware of any economic commentator in the media that has drawn attention to the OBR's forecasting shortcomings. Indeed, far from it, there seems to be a willingness amongst the UK media to accept all pronouncements from the OBR as gospel and, in Westminster, a collective blind faith in its prognostications. Indeed, some politicians plan on giving the OBR more power to independently publish its impact assessments of any major tax and spending decisions. Until someone has properly analysed the OBR's forecasting framework, I doubt this would be well advised.

My bet is that Einstein would observe that it would be unwise to rely on an organisation that had consistently failed to accurately forecast the performance of the UK economy in the past to continue to dictate how the government should frame its fiscal policy decisions in the future. In fact, my read of his personality is that he would have described it as madness.