GRAHAM BARR AND BRIAN KANTOR: Reserve Bank muddles its models

Scientific advances have driven extraordinary technological development and associated human wealth; think improvements in transport, production, agriculture and medicine. The recent development of artificial intelligence programs such as ChatGPT and Midjourney has shown that even the computer chips contained in cellphones can generate written prose which mimics the style of Shakespeare, or paintings in the style of Van Gogh.

However, the application of science to modelling human behaviour, and to related areas including economics and financial markets, has been less successful.

Recent interest rate hikes by the Reserve Bank have attracted lots of criticism, and we have argued in this publication that they were an unfortunate mistake. Now, the Bank uses an econometric forecasting model as a central lever of input into its economic forecasting, and for formulating its monetary policy.

The forecasting model is used, for example, to generate forecasts of economic growth, as well as the expected impact of interest rate changes on the economy. The efficacy of the model is thus a key driver of the Bank’s monetary policy and, through the implementation of that policy, of the path the economy will take.

The principle behind all model-building is to use data to estimate the structure of interrelationships between variables, and to measure how that affects particular variables of interest (generally called the dependent variables).

Ideally, you would have a model structure where all the variables, apart from the dependent variables, would be under your control. For example, if you wanted to measure the impact of different fertilisers on the growth of potato plants for a set of climatic conditions, you could have multiple experiments with different types of fertiliser under controlled conditions and then measure the different growth rates produced by different fertilisers.

Such experiments are repeatable so that in addition to estimating the efficacy of different fertilisers, the variability in outcomes could also be assessed. This would mean that you could measure both the average effectiveness of a particular fertiliser in promoting growth and the reliability of the effectiveness of fertiliser.

We cannot, by contrast, run experiments in the economy, and we certainly can’t put the clock back and run them again. So the only thing an economic model can do is to say: this is what happened to growth rates in the past when certain generally unforecastable things (such as the commodity cycle and the political environment) occurred. Then, if we make the correct assumptions about those things going into the future, and the economy behaves in the same way it has in the past, we can assume our forecasts of growth rates will be accurate.

The set of assumptions made by the Bank is wide-ranging and covers those variables you feel are entirely beyond its control, such as the GDP of trading partners, the oil price, the state of the commodity cycle, world food prices and the local electricity price. It would also include, in each case, the Bank’s assumed interest rate response to those shocks.

For forecasting purposes, the Bank then makes a set of plausible assumptions about the values of these variables in the future and computes the economic variables of interest, such as the forecast growth rate. Therefore, the Bank’s forecast for the growth rate is completely dependent on the assumptions it makes about the other variables in its model. Even if the model is close to reflecting reality, the growth rate forecast can only be accurate if the assumptions made turn out in practice to be true.

By running the model with different sets of plausible assumptions, the Bank can generate different growth (and other variable) forecasts and thus obtain some measure of the uncertainty attached to these forecasts. In the latest biannual Monetary Policy Review published in April 2023, for example, the Bank publishes forecast confidence levels for inflation and real growth for levels of certainty ranging from 10% to 70%. Statisticians normally use confidence levels of 95%, so the ones used in the Bank model indicate very low levels of confidence.

The forecast chart for growth rates indicates that the actual South African growth rate by end-2024 has only a 70% chance of falling between -4% and +3% (a 7% range!).

In reality, this implies the model has very little useful information to convey about what the real growth rate at end-2024 will be at all.

The Bank’s modelling of the South African economy and the associated forecasts may have perceived scientific validity but instead represent an inadequate, even misleading, basis for managing the economy, given its unpredictable nature.

Exchange rate shocks of the kind periodically suffered by South Africa are not predictable and do not easily fit into any model which relies on consistent cause and effect.

Interest rate changes do not help an economy recover from an exchange rate shock; in fact, they make it harder to do so. Monetary history, not any forecasting exercise, indicates clearly that central banks should simply ignore these shocks; it is not responsible for them, nor can it help overcome them.

* Barr is emeritus professor of economics and statistical sciences at the University of Cape Town; Kantor is emeritus professor of economics at UCT and head of the Investec research institute