2^nd Workshop on Short Range Ensemble

7-8 April 2005, Bologna-Italy

Participants

Acordon Valentina, Amici Maura, Barbiero Roberto, Barkmeijer Jan, Bassini Sanzio, Boccanera Francesco, Bonavita Massimo, Bowler Neill, Buizza Roberto, Cacciamani Carlo, Capaldo Massimo, Cesari Davide, Charron Martin, Chessa Piero, Corazza Matteo, Davolio Silvio, Denhard Michael, DiGiuseppe Francesca, Elementi Marco, Ferri  Massimo, Fornasiero Anna, Frogner Inger-Lise, Fruewald Dieter, Garcia-Moya Jose, Hagel Edit, Hayes Patrick, Hohenegger Cathy, Homar Victor, Ivatek-Sahdan Stjepan, Jensen Marit Helene, Jongen Suzanne, Kallberg Per, Keil Christian, Kilpinen Juha, Kok Kees, Latinne Olivier, Marsigli Chiara, McQueen Jeff, Mezzasalma Paolo, Milelli Massimo, Montani Andrea, Mylne Ken, Nemegharire Jean, Nicolau Jean, Paccagnella Tiziana, Papetti Paola, Piazza Andrea, Prezzi Paolo, Pristov Neva, Quiby Jean, Quinet Alfred, Romero Romualdo, Rotach Mathias, Santos Carlos, Sasu Monica, Sattler Kai, Schumann Thomas, Sorani Roberto, Tan Yan, Tibaldi Stefano, Torrisi Lucio, Trepte Sebastian, Trovatore Elisabetta, Valentini Andrea, Vannitsem Stephane, Walser Andre', Wang Yong, Wei Mozheng, Wichers-Schreur Ben and Wilson Clive.

1. Agenda and presentations
2. Workshop Report
3. Highlights of the final discussion

1. Agenda and presentations

2. Workshop Report

J. Quiby (with the help of the notes of Michael Denhard)

Participation

21 participants representing the National Weather Services of the following 15 nations: Austria, Belgium, Croatia, Denmark, Finland, France, Germany, Hungary, Italy (ARPA Emilia-Romania), Netherlands, Norway, Slovenia, Sweden, Switzerland, United Kingdom.

Better graphical representation

Products: Probability of maximum 10m wind gusts.

Only 8 NWS send wind gust forecasts. Thus the PEPS area (35N-70N, 30W-30E) cannot be entirely covered with this information.

On the plots, an area without information or an area with a 0% probability is uncoloured, which makes them undistinguishable.

This has to be changed: the area with 0% probability must be coloured and the area with no information must remain uncoloured.

Precipitations

Models give in general too much precipitation along their lateral boundaries, i.e. in the relaxation zone. For 4 models, the relaxation zone is not considered. The point whether it would not be better to ignore the relaxation zone for all the models must be examined.

Minimum number of models for a forecast (mean or probabilistic)

Today, at grid points of the PEPS grid covered with 2 models only, mean and probabilistic forecasts are computed.

This is maybe acceptable for the mean forecasts, but not for the probabilistic forecasts.

It has been requested that probabilistic forecasts should only be computed where there is a minimum of 4 models.

Calibration of the model ensemble

The forecast ensembles should be calibrated and this will be done by the DWD with the use of the Bayesian Model Averaging (BMA) technique (see presentation of Michael Denhard).

Statistics of the model availability

It has been requested - and indeed it would be interesting - to have a monthly or at least a quarterly statistics of the mean number of models available for each point of the PEPS grid, at least for the 00 and 12 UTC integrations.

Extension of the forecast range

In today's PEPS forecasts, the lead time does not exceed +30 hours. The wish has been expressed to also have forecasts (mean and probabilistic) for the second day, i.e. for the period +30/+54 hours:

• 24 hour (+30/+54) accumulated total precipitations
• 24 hour (+30/+54) accumulated snow
• Maximum wind speed for the period +30/+54
• Maximum of the wind gusts for the period +30/+54

Presently, PEPS receives forecasts with a lead time of 54 hours or more from 6 models only. Several models of the 23 models participating are integrated operationally only up to 48 hours or less. The first step in this development is to find out how many models would be at disposal with a lead time of 54 hours.

Verification

With time passing, the need for a verification of the PEPS forecasts becomes always stronger. Unfortunately, the DWD does not have presently the capacity to realise a comprehensive verification of the whole PEPS area: it will limit itself to a basic verification. DWD will contact KNMI for the definition of the verification strategy. Besides this, each participating NWS should try to thoroughly verify the PEPS forecasts over its territory by using its high resolution observations.

Other variables

Michael Denhard has asked whether there is a need (next to total precipitation, snow, wind speed and speed of gusts) for other variables. This does not seem to be case presently. As the PEPS Project puts emphasis on disrupting and dangerous weather, the next variable could be a convective index.

Neighbourhood Ensembles

For each grid point of the PEPS grid, probabilities could be based not only on the values given by the different models, but also on the values at grid points in a neighbourhood. In this case, the resulting probabilities would be results of much larger ensembles.

PEPS data for research

For research works by NWS participating to the Project, PEPS results can be obtained from the DWD. If it would be necessary to also have individual forecasts, the DWD will have to ask the producers of the forecasts whether it can deliver them to the NWS wishing them.

Other Institutions - as Universities - wishing PEPS results or individual forecasts must ask for them through the NWS of their country. Individual forecasts can only be used for research works directly connected with the PEPS method.

Commercialization of the PEPS forecasts

As it is very probable that, soon or later, there will be opportunities to sell PEPS forecasts, Jean Quiby encouraged the participants from NWS members of ECOMET to go to their ECOMET Officer and inform him or her of this possibility.

As ECOMET did it for the radar composits, this organisation should develop formulae for the determination of the prices of the different PEPS forecasts as well as for the computation of the financial returns for participating NWS.

3. Highlights of the final discussion

report (reviewed and supplemented by Roberto Buizza), J. Quiby SRNWP Programme Manager

Chair: Roberto Buizza

As the workshop final discussion was a free discussion not following a plan, the following summary is to be seen as a list of the highlights of the discussion. It must be understood that the perception of what is a highlight is very subjective and personal. Thus nobody will claim that the following summary contains all the important statements and ideas expressed.

Quality of probabilistic forecasts

With the EPS technique, we could be tempted to think that it is no longer imperious to improve the different aspects of the classical NWP, as the data assimilation, the model physics or the model dynamics. But Roberto Buizza was very clear: it is an illusion to think that we could make good probabilistic forecasts with bad analyses and bad models.

Bad analyses and bad models could enlarge the spread, but the spread must be centred over the true solution, over the reality. It has even been said that "the quality of the data analysis and of the model are more important than the way perturbations are created".

The Breeding Method

This method is loosing momentum. At NCEP it is already decided to abandon it.  Its main advantage was surely its relative practical simplicity, but its theoretical background is lean which does not allow a good control of the perturbations, its main deficiency being that there is no control on the orthogonality of the perturbations.

DA and EPS

If you admit - and the trend seems to go into this direction - that the future of numerical forecasting will be given by systems able to explicitly account for uncertainties at the analysis and as well as at the forecast times, you marry DA and EPS with each other. The new problem is to choose them such that they are consistent with each other.

Ensemble Data Assimilation (EDA)

Work in Europe on EDA is starting to take momentum, with the Met Office experimenting the use of EnKF techniques to generate ensemble initial perturbations, and ECMWF starting to revisit earlier work that tested the use of ensemble data assimilation to generate ensemble initial conditions.

Ensemble Kalman Filter and Ensemble Transform Methods

The developments around these methods are presently booming in America, but not so much in Europe. The difference is striking: almost the whole European effort is devolved to the variational methods (with 4D-Var as final aim) when in the United States both paths - Var and KF - are followed.

The classical KF method has undergone several developments recently leading to the ETKF (Ensemble Transform Kalman Filter) and the LEKF (Local Ensemble Kalman Filter).  The NCEP ET (Ensemble Transform) has been praised by Mozheng Wei as being the best choice.

Singular Vectors

For the global models: Today bred-vectors are still more largely used than SV-based methods. But SVs are becoming more popular. For example, Environment Canada and the Met Office are starting to experiment their use in ensemble systems. It is very probable that the SV method will supplant in the future the bred-vectors.

For the LAMs, The general opinion was that this method is not yet "ripe" for use with the high resolution LAMs.

Perturbation incompatibility in LAMs:

If you choose to perturb simultaneously analysis and boundary conditions in the way that each member will differ by its initial condition and by its boundary conditions during the forecast, you have to be careful that these perturbations are compatible.

It has been said that it would not be the best - at least from a theoretical point of view - to perturb both independently, for example to produce the analysis perturbations from random perturbations of the observations and to take the boundary conditions from members of an independent global EPS.

It has been claimed that the ideal LAM EPS would be produced by a unified model, that is with a LAM very similar to the global model where the BC would come from and that each LAM EPS member would be forced by a unique corresponding member of the global ensemble: initial condition and boundary conditions all from the same integration of that global member.

With this scheme, each LAM EPS member would perform a pure dynamical downscaling of its corresponding member of the global EPS.

Where to put the uncertainties?

Ensembles are today primarily defined by putting the uncertainties in the analyses. Much less work has been done on the model uncertainties. One colleague expressed the wish that before we put a lot of effort into the model uncertainties, it would be valuable to have an idea about the respective share in the probabilistic forecasts between the model uncertainties and the uncertainties in the initial conditions.

But the problem is not easy because the analysis is model dependent. Therefore the analysis errors are influenced by model errors. This inter-connection makes it extremely difficult to clearly separate the role of initial and model uncertainties.

At the workshop there has been very little discussion on stochastic physics. The main contribution in this field has been in the presentation of the Meteorological Office with the random use of different coefficients for some physical schemes as convection, boundary layer and gravity wave drag.

How to account for the intrinsic model uncertainties?

One of the basic intrinsic uncertainties in a numerical model is due to the numerical discretisation in space and time. This uncertainty should be dealt with by the use of pure stochastic perturbations.

The use of different physical parameterizations between members (for example some members with mass-flux, others with Kain-Fritsch, others with adjustment) does not account for the intrinsic model uncertainties. This technique can enlarge the spread, but favours clustering of the members around the different parameterizations.

When parameterizations are randomly changed in the course of integration, the different model configurations are not all of the same quality: to enlarge an ensemble by introduction of "second quality physics" cannot be a strategy for the long term.

Ideally, we should have true stochastic parameterizations instead of randomizing parameterizations.

Ensemble size for a LAM EPS

This very important question has been asked by the Chairman, but raised up in the audience only one reaction. It must be concluded that we do not yet know the answer.

But we have to be careful not to continue to think "the bigger, the better". In the ECMWF presentations, it has been shown that there is a threshold in the number of members above which the improvement gained by adding members is marginal. For Robert Buizza a limited number of good members is better than a large number of average members.

Use of short-range LAM EPS

It has been generally admitted that in the Weather Services forecasters know today what the method is about and - more important - begin to use it when looking for information when confronted with an uncertain weather development.

In Europe, no Weather Service issues today weather forecasts in probabilistic terms. But in the French speaking part of Switzerland, the TV and a major newspaper accompany each forecast - presented in deterministic terms - with a confidence index (between 10 and 1) based on a clustering algorithm applied to the ECMWF EPS.

But it seems that the general public is no yet ready to accept probabilistic forecasts.

Franco Molteni indirectly confirms that by indicating that neither Meteo-France nor the Meteorological Office present probabilistic forecasts on their respective web site (also true for the DWD).

Collaboration

It was a wish of Tiziana Paccagnella that we discuss (and find!) possibilities of collaborations. The SRNWP Programme Coordinator thought that it would be wrong to define at the end of the meeting a deep, large-scale collaboration. This would very probably be unsuccessful. More promising in his view would be to start with something small and to enlarge it if it suscitates interest.

It has been decided to create a chapter on "Short-range EPS" in the web site of the SRNWP Programme. As first step, information will be collected from the NWS active in short-range EPS in order to give a synoptic view on the state of this technique in Europe.

As second step, we should try to agree on some standard products and some verification scores.

It will be rather easy in my view to find agreement on the scores (Brier, ROC and Talagrand) but the very different spatial resolution of the European EPS will still make comparisons delicate.

Connection with THORPEX-TIGGE

In view of the importance attached by the First TIGGE Workshop on "the interfacing of global ensemble prediction systems with LAM ensembles", the Workshop has proposed that the Executive Director of the THORPEX/WMO Project (Dr  Burridge) should be informed that the European Short-Range EPS Community strongly supports TIGGE activities and thinks that it should be represented at any forthcoming planning meeting when appropriate.

The Manager of the SRNWP Programme will write to Dr. Burridge on behalf of the Workshop participants.

Next SRNWP Workshop on Short-Range EPS

When closing the meeting, Massimo Capaldo announced on behalf of the UGM that the next workshop will take place in Italy in the spring 2007.