Understanding SA’s Covid models: 6 key points
It’s encouraging that scientists are being honest about the limitations of their models
Last week, health minister Zweli Mkhize convened a virtual meeting with the media in which a number of scientists released their models in a deluge of data.
Some of these models were created for the government, others were created by mathematical groups with an interest in understanding the epidemic’s impact on SA. After months of secrecy around the models used to predict the path of the virus in this country, the open meeting was a welcome change.
But the problem is that the government is not saying which models it is using, or how they shape the decisions it is taking on the lockdown.
Nonetheless, there were some overarching themes from the range of complex projections from economists, actuaries, statisticians, mathematicians and epidemiologists. Here are the six key ones we identified.
1. We don’t know what we don’t know
“Uncertainty” was mentioned often during the presentations, which seems fair since the models rely on data that is often changing or missing.
Actuary Barry Childs, who chaired the group that built the official Actuarial Society of SA model, which predicts 48,300 deaths this year, said there were “concerns” globally with data that is incomplete but is used to shape models.
For example, some scientists in the UK and elsewhere have suggested that the official death statistics from Covid-19 may be too low as officials are not correctly categorising the cause of deaths in old-age homes or private homes.
Equally, Prof Juliet Pulliam, of the SA Centre for Epidemiological Modelling & Analysis, emphasised the “uncertainty” of the centre’s long-term models. The SA Covid-19 modelling consortium, which she is part of, predicts 13-million infections in SA by November, though 75% of those people will have no symptoms.
However, this seems remarkably high — more than double the total known 5.4-million infections across the world. But as Pulliam put it, “the longer ahead the model looks, the more uncertain projections become”.
Childs expressed sympathy with leaders facing the “difficulty of making significant decisions … amid such uncertainty”.
2. There has been robust criticism of models
While many people question the extraordinarily high projections of 13-million infections in SA, they’ll be pleased to know the experts are grappling with these questions too.
Wits health economist Prof Alex van den Heever was blunt in his presentation, saying: “I do not trust forecasts.” Van den Heever has suggested, for example, that many models don’t sufficiently take into account how changes in people’s behaviour might reduce infections.
Childs admitted that the official actuarial model had faced an “extraordinary wide range of critiques”. For example, some actuaries told his team their figure of 48,000 deaths is “four times too high”, while others said it’s “four times too low”.
It’s a vexed issue: some experts don’t see why the SA models predict higher fatality rates than other countries with older populations which have been hard hit by the virus.
3. There’s honesty about the limitations
It’s also encouraging that scientists are being honest about the limitations of their models.
In speaking about his actuarial model, Childs spoke of how it was, in some senses, “quite simplistic” as it didn’t take into account population density and varying susceptibility to the virus. In crowded areas, the virus may spread faster, while some individuals, like children, appear to be less likely to catch the virus.
Mkhize also weighed in on the limitations of models: “I don’t think there was a model upfront able to predict what’s happening in the Western Cape. One model I saw predicted Gauteng exploding first, followed by KwaZulu-Natal.”
This proved devastatingly incorrect. Of SA’s 22,583 confirmed infections, at last count, the Western Cape’s 14,740 infections accounted for 65% — more than five times the level in the country’s economic heartland, Gauteng.
This isn’t just a testing issue either, since the Western Cape also accounts for 65% of SA’s 429 deaths so far. The 281 deaths in that province are nearly five times more than the fatalities in the next worst province, the Eastern Cape, which has 59 deaths.
But the encouraging part is that models are being updated all the time as new information emerges.
Childs said the actuarial society is recalibrating its model with new data, including the local and global data on deaths.
Ashleigh Theophanides, an actuary at Deloitte, said her projections are “updated on a developing basis as new information arises, in order to be as accurate as possible — this is particularly true of forecasts which are more than one month in advance”.
4. There’s more to the models than just the deaths
At least three models predict between 40,000 and 50,000 Covid-19 deaths in SA this year. These models are important since they could, in theory, help the government plan for more intensive care and hospital beds.
But there are other models with different objectives that should inform SA’s response to the crisis, such as measuring how SA can reduce the spread of the virus.
Co-Pierre Georg, an associate professor at the University of Cape Town’s African Institute of Financial Markets & Risk Management, said a society that differs in wealth and living conditions required different models to reflect likely differing levels of disease spread. “A heterogeneous society requires heterogeneous agent models,” he said.
Georg and colleagues, working with the Massachusetts Institute of Technology, have built a custom-designed system for cellphones to improve contact tracing.
Equally, Van den Heever’s model looks at the effectiveness of various interventions — social distancing, hand washing and mask wearing — as well as factors such as warm weather in “suppressing the epidemic”.
His initial model calculated that if the lockdown was to really cut down infections, it would need to take the form of a hard lockdown for 193 days — which clearly wouldn’t be practical, since it would decimate the economy. As a result, said Van den Heever, “we need alternative strategies to suppress the epidemic”.
Zaid Kimmie, a statistician and mathematician at the National Institute for Communicable Diseases (NICD), explained that he was using spatial modelling to measure the movement of people between wards in the Western Cape using cellphone data. “This gives us a sense of what effect movement is having on the spread of the virus,” he said.
This particular model is used to predict where hotspots will arise, to allow for quick detection of infections in small areas.
But Kimmie warned that the spatial model was a work in progress. While it isn’t good at predicting in exactly which ward the virus will spread next, it does a pretty good job of predicting 21 days in advance in which wards it will occur.
5. Let’s not forget the 1,300 other deaths per day
There is a reassuring awareness from scientists that there are other diseases to also think about, and the danger is that a singular focus on Covid-19 could worsen fatality rates.
Childs pointed out that globally, there is “significant concern that management of other conditions will fall by the wayside”.
He reminded people that in SA there are still 70,000 deaths from HIV/Aids each year, even if some of them are reported on death certificates as pneumonia or tuberculosis.
And that’s besides the 100,000 deaths which SA faces every year from noncommunicable diseases, including heart disease, strokes, aneurysms and some cancers.
Already, the World Health Organisation (WHO) has warned of an increase in HIV, TB and malaria deaths in Sub-Saharan Africa over this time, as people risk not getting properly diagnosed or aren’t able to access chronic medicines.
6. There are voices to keep mathematicians rooted in reality
Nick Hudson, an actuary who founded a group consisting of statistics, economists and actuaries called Panda (Pandemics ~ Data & Analytics), was one of those who presented his conclusions last week to Mkhize and the other scientists.
Hudson urged mathematicians to make sure their models weren’t divorced from reality and the rates of death and infection globally. “We have to be careful not to use models that are inconsistent with the reality that has emerged elsewhere in the world,” he said.
Hudson also warned of the danger of singularly focusing on Covid-19, saying that the continued lockdown would cause a “vast humanitarian disaster”.
“We have to remember that poverty kills as well as viruses,” he said.