You may have heard the jokes. One of the founders of modern economics, Paul Samuelson, once observed how "the stock market has predicted nine out of the last five recessions". To which the New York Times, plus a hoard of others, have added, "And economists have an even worse record". Yes, thanks for making fun of my profession, often referred to as the "dismal science". And on that note, let's reference the government's Covid-19 modelers who're trying to predict virus numbers to inform lock-down decisions.
A year ago, Newshub reported that "A scientist [Shaun Hendy] whose work helped the Government decide when to lift lockdown levels has criticised a report which suggested lengthening the time spent in level 4 cost a fortune, while providing few health benefits". That report was by the Government's own Productivity Commission. It was a cost-benefit analysis which tried to work out whether a decision to extend a lock-down in 2020 was justified on a social welfare basis. Hendy said the Commission's work was nothing more than a "toy model".
Interesting. Since just over a week ago, Stuff ran the headline "Covid-19 NZ: Why are scientists saying there are 50 to 120 cases in the Delta outbreak". Those scientists are part of the group led by Hendy. When virus cases exceeded 120 within a short time of their 50-120 case prediction, that same group of modelers changed tack. A very different set of predictions were wheeled out, this time with 200 cases as the "best case scenario" and up to 1,000 as the worst. Yet within barely a day or two of making this new prediction, numbers already exceeded their revised "best case scenario". So will they revise their revision?
Of course, we're all super grateful to these modelers for applying mathematical techniques with the earnest intention of trying to help inform debates. But wouldn't it have been better not to make any "scientific predictions" of case numbers when so much is still unknown, when there is still so much uncertainty, about the nature of the current outbreak? Without such information, it's extremely hard to correctly parameterize their models.
Back to the Productivity Commission. It was doing exactly what it should be doing. Trying to work out the full costs and benefits of government policy. And those costs and benefits play no role whatsoever in the Covid-19 mathematicians' model which is narrowly focused on simply trying to predict case numbers.
Sources: