When your model doesn’t match reality, which do you choose to revise? This was the difficult question those at the Bureau for Economic Research (BER) must have asked themselves in early May as analysts across the country awaited the outcome of the purchasing managers’ index (PMI).

This globally recognised survey-based index is regarded as an accurate, timely gauge of business trading conditions. SA’s PMI is released slightly later than those of several other economies, and market participants had watched as lockdown measures devastated sentiment across the globe. Manufacturing PMIs for the UK, eurozone and US fell in April 2020 to record multiyear lows around the 30-35 index level.

Supported by endless signs of damage to the local manufacturing sector, it seemed that, such as the coronavirus, horrific PMI data was spreading inexorably towards SA. And then, surprisingly the BER headline PMI for April came out at a rosy 46.1 points, down just marginally from the 48.1 points seen in March and actually an improvement on the 44.3 points seen in February.

The PMI can fluctuate between zero and 100, where 50 is the threshold between expansion and contraction of business activity. A PMI score of 46.1 therefore implies a modest contraction in the manufacturing sector. Were SA’s manufacturers really doing far better than anyone could have anticipated? Unfortunately, not. Rather, Covid-19 has revealed a broken PMI.

The BER’s PMI (there are alternatives, but the BER’s is the eminent one) is a single index built from the results of five weighted questions sent out to senior decisionmakers at a representative group of manufacturers across the country. These questions pertain to business activity (production), new sales orders, employment, supplier deliveries and inventories.

Four of the five subcomponents have collapsed dramatically during lockdown (much as would be expected), to their lowest levels. Business activity, which attempts to measure the level of overall business output, collapsed to just 5.1 points in April, from 44.6 points in January — intuitively reflective of the economic crisis facing the country. But the supplier delivery component went screaming upwards to its highest recorded level.

The simple reason the overall PMI scored so high in April is that the rapidly improving supplier deliveries component has the largest weighting in the PMI — it almost perfectly offset the dramatic decline of all the other components. This is anomalous — supply chains are in dire straits. To its credit, the BER has made this clear in its media statements surrounding the PMI release. Its website now directs headline PMI to the business activity component instead.

But there is far more to this. Why did the supplier delivery component shoot higher rather than break down under lockdown? And why is it weighted so heavily (40%) against the far more intuitive business activity component (5%)? Suppliers delivery attempts to measure the availability/obtainability of input materials, goods, and services. The simplest economic interpretation would be that when suppliers’ delivery capability gets worse, it puts pressure on manufacturing businesses and the PMI should fall (all else being equal).

No business wants its supply lead times extended, deliveries delayed, or orders less quickly processed. Instead, this component enters the BER PMI in the reverse direction. A deterioration in the obtainability of input supplies is actually regarded as good for business. In doing things this way, the BER has adopted international methodology based on a demand-biased Keynesian understanding that tighter supply must result from higher demand (making suppliers busier and testing their capacity). This is taken to reflect more economic activity, and therefore the PMI improves (all else being equal).

The rationale behind this methodology is likely based on mechanistic econometrics. So well did suppliers’ delivery correlate with previous estimated cyclical fluctuations that as recently as October 2019 the BER reweighed this component in the headline index from just 15% to a dominant 40%.

But Covid-19 global lockdowns have exposed the flaw in such a mechanistic approach. It is obvious that while poorer supplier performance may sometimes be a sign of rampant demand, it is more likely a sign of poorer supply. A sign of imminent disruption, not industrial nirvana. Lockdown has created one of the biggest supply crises in history, and the PMI model thinks this is wildly bullish.

ETM Macro Advisors. in collaboration with Econometrix, has reworked the headline PMI using what we believe to be sounder and more intuitive economic logic: supplier deliveries now enters the headline PMI in the inverse to the BER version, for reasons stated above.

  • The employment component is removed since the goal of production is not to maximise employment but the market value of products at minimal costs.
  • Likewise, we remove inventory stocking, which can be ambiguous in their signal. While increased buying of raw materials would often be a bullish signal, it could at times reflect looming inflation fears as manufacturers buy abnormally large quantities of stock to hedge against expected losses. Rising inflation is typically associated with a looming recession and demand destruction, meaning these stock purchases could fail to result in good final sales.
  • Finally, we add in a component that the BER collects but does not include in the headline PMI, the prices component. This is the prices producers pay for their inputs, and therefore should also be entered into the model in the inverse to how it is reported — rising inflation in inputs is a negative factor for manufacturing companies.

Taking these changes into account, we present the modified PMI below alongside the original. Of course, the components in our version are still those generated by the BER survey, and therefore credit must go to the BER for collecting this useful data. Whereas the BER PMI fell to just 46.1 in April and increased to 50.2 in May, our version crashed to just 14.6 in April and only recovered to 40.2 in May. The real picture is one of a manufacturing sector in big trouble.

Aside from the fact that the ETM version more accurately reflects the Covid-19 lockdown crisis, it also does better at capturing and predicting recession periods (highlighted in grey). It reflects the SA manufacturing slump of the past decade more clearly and shows a more pronounced and realistic expansion during the 2003-2006 boom period. We think it tells a far more accurate story of the past 20 years of SA manufacturing and is often meaningfully at odds with the official headline PMI.

Note also how the ETM version was already flagging a sharp decline in February 2020 due to diminished demand and supply disruptions from Asia. The official PMI had not registered this at all.

We want to emphasise these model misspecifications are by no means unique to the BER. PMIs are misspecified globally as “best practice”. More generally, as data becomes abundant there has been a concerning tendency for analysts to allow data-fitting to override sound economic logic.

This does not invalidate an econometric-empirical approach to economics. Rather, it highlights just how important it is that such an approach is properly framed by sound logic. The famous economist Ronald Coase stated it aptly: “If you torture the data long enough, it will confess to anything.”

As the world becomes swollen with useful data, those whose livelihoods depend on using such data to make decisions can become ever more insensitive to what they are really looking at. What does a particular data abstraction or aggregation represent about the real world? Is it telling me what I think it is? Is it built on sound foundations?

In considering these questions, we also answer the first one we posed — you revise your model to match reality. Reality is far less forgiving of your misinterpretations.

• Dinham is senior economist at Econometrix, and Lamberti founder and strategist at ETM Macro Advisors.