Contributed by Anders Persson, a HEPEX guest columnist for 2014
It is true that computer made deterministic weather forecasts (ECMWF’s T1279, GFS, ALADIN etc) must be physically realistic. One of the tasks of the monitoring at ECMWF is to make sure that the deterministic forecasts are physically realistic. A 10-day forecast map should appear as realistic as a 1-day forecast map or an analysis.
But post-processed, refined or “tailor suited” deterministic weather forecasts do not have to by physically realistic. One of the major problems in today’s weather forecasting is that this distinction is not understood or upheld.
Assume the deterministic NWP model predicts that low clouds will clear and the 2 m temperature will drop from +2°C to -4°C (figure 1). The forecasters are now supposed to justify their existence by “adding value” to the NWP by modifying it using their experience, physical understanding or knowledge about NWP models.
This works well when the NWP model has clear systematic errors, but becomes more and more difficult with the steady model upgrades. But the forecasters can do something else: they can question the clearing itself!
They might, for example, assign only a 40% probability that the clouds will disperse and the temperature drop to -4°C and assign a 60% probability that it will stay cloudy with +2°C. Weighted together this yields -0.4°C (or rounded off to ±0°C) as a “most likely” or, if you want, “most tactical” temperature forecast. This is the ideal number to put in the verification form.
The ±0°C forecast might also be given to the public, via newspapers, radio or TV stations. But the consequences for the public of a temperature drop below zero are often greater than if it stays mild. For this reason, the forecasters might tweak their ±0°C forecast into -1°C (or perhaps even -2°C).
None of these ±0°C, -1°C or -2°C values will never verify (or only for a very short time), it will either be +2°C or -4°C. The forecasts are indeed unrealistic. But their quality does not lie in “how very good” they are, but “how little bad” they are. Instead of trying to make the 100% perfect prediction, the forecasters have tried to minimize the expected forecast errors. By doing this, they have not acted as physicists but as “intuitive statisticians”.
What the forecasters did was for the ±0°C forecast they used (intuitively at least) an optimal least square approach in order to minimize the Root Mean Square Error (RMSE). This approach will in the long run beat the deterministic NWP!
The RMSE assumes implicitly a symmetric cost or penalty function: negative errors are seen as bad as positive errors (figure 2, left). In this case RMSE regards it as equally bad to miss the frost as to issue a false alarm. The RMSE is “objective”.
When the forecasters issued the -1°C or -2°C forecasts they (intuitively at least) applied an asymmetric cost function (figure 2, right) where “missed events” (forecast too warm, i.e. positive errors) are considered worse than “false alarms” (forecast too cold, i.e. negative errors). This deviation from “objectivity” will be slightly punished by the RMSE, but still not as much as for the +2°C or -4°C forecasts where RMSE will be either zero or very large.
But this approach seems to have catastrophic consequences.
As an example, I use the the same scenario as in my reply to Tom’s comment: Ships leaving the Swedish waters heading for North America have to make a choice of either passing north of Scotland or through the English Channel.
Mostly, the meteorologists involved in ship routing can give them good advice at an early stage. But, as one of my critical colleagues once pointed out: if the weather situation is tricky and the odds are equal for either route – would the ships then, by a meteorologist following your philosophy, be sent on an intermediate route straight into Newcastle harbour?
I was really floored by this objection and saw my whole forecast philosophy in tatters. It would not only result in physically unrealistic statements, but hundreds of sunken ships and drowned sailors in Newcastle harbour!
But Jan-Erik, one of our most experienced ship routing meteorologists, told me: -This is exactly what we would do in such cases – we direct the ships towards Newcastle. We would, however, do this as a temporary measure, for the captains to await later forecast information. Switching from some “middle course” route to one of the “realistic” routes is on average less costly than to switch from one extreme to the other (figure 3).
The problems mentioned above will evaporate if the forecasts are issued in probabilistic terms. However, traditionally educated meteorologists will of course find these even more “physically unrealistic”. As one told me: -We don’t live in a quantum world like Schrödinger’s cat, but in a world where there is either “frost” or “no frost”, never “40% frost”.
So there are more issues to come back to . . . .
Next post: 18 July 2014.
Anders will be contributing to this blog over the year. Follow his columns here.