This webcast is called “Introduction to NAEFS: The North American Ensemble Forecast System”. My name is Dr. Bill Bua, and I’m a project scientist at the University Corporation for Atmospheric Research’s COMET® Program.

This first section of this webcast briefly reviews ensemble forecast systems in general, that is, why we use them; and the North American Ensemble Forecast System, and its architecture in particular.

At the end of this webcast, you will be able to …

1. Recall why we use ensemble forecast systems (EFSs)
2. List the EFSs the North American Ensemble Forecast System (NAEFS) is built from
3. List the characteristics of the EFSs used to create NAEFS;
4. Explain why NAEFS performs better than the two EFSs comprising it;
5. Find NAEFS products and data for downloading or viewing on the internet, and
6. Use NAEFS products effectively in forecasts for your local area.

We’ll briefly review the “what” and “why” of EFSs. Then we’ll describe NAEFS and its component EFSs, and how NAEFS performs relative to those components.

We use EFSs to estimate the uncertainty in a forecast.

Using different NWP models and/or different initial conditions gives an array of plausible forecasts, especially farther out in forecast time. The only limit is the amount of computing power available.

The operational Global Forecast System (GFS) is used to create the GEFS. Because of computational constraints, the ensemble GFS is run at about 125 km resolution, rather than the operational 45-km resolution. As of April 2009, model uncertainty is ignored.

GEFS is run every 6 hours starting at 00 UTC, with 20 forecasts, or ensemble members, made by creating 20 initial conditions using the Ensemble Transform Bred Vector method. A low resolution control is run from initial conditions used in the operational GFS, but at EFS resolution. All are run to 16 forecast days to support the NCEP Climate Prediction Center’s 6-10 day and week-2 probabilistic forecasts.

The other EFS in NAEFS, the Canadian Meteorological Center (CMC) EFS or CEFS, uses CMC’s Global Environmental Multiscale (GEM) model at about 100-km horizontal resolution.

Unlike GEFS, model uncertainty is included in CEFS. GEM has 24 combinations of physics schemes. Additional physics uncertainty is provided by randomly adding or subtracting up to 50% of the calculated effect of model physics on forecast variables.

This graphic shows fall 2007 root mean squared error (RMSE, solid lines) for ensemble mean and spread or standard deviation (dashed lines) for Northern Hemisphere 500-hPa heights. GEFS is black, CEFS red, and NAEFS blue. The y-axis is RMSE, and the x-axis is forecast days.

Theoretically, NAEFS *should* perform best because it has 42 versus 21 ensemble members, and multiple models. CEFS should also perform better than GEFS. But, before discussing the verification, keep in mind a “rule of thumb” for EFSs: the standard deviation or spread should approximately equal the root mean square error (RMSE) of the ensemble mean. If it is less than the RMSE, too few extreme events will be forecast; if it is more than the RMSE, too many extreme events will be forecast.

Note:

1. Ensemble mean RMSE is similar for CEFS and GEFS, but somewhat less for NAEFS. At day five, ensemble mean NAEFS has the same forecast skill as GEFS and CEFS forecasts did 12 hours earlier, a ½ day gain in skill!
2. GEFS has RMSE less than its spread. Meanwhile, CEFS and NAEFS spreads and RMSE are nearly equal.

So during fall 2007, NAEFS had more accurate ensemble mean forecasts than CEFS or GEFS, and better captured possible forecast outcomes than GEFS, based on our rule of thumb.

Another graphic for 500-hPa heights for fall 2007 over the Northern Hemisphere shows the mean absolute value of the error (MAE) in dash, and ensemble mean error or bias in solid color. Color coding is the same as in the previous slide. The MAE for GEFS and CEFS are approximately equal; small improvements are seen in NAEFS.

However, the biases show the power of combining different models into one EFS. CEFS high geopotential height bias combined with GEFS low height bias of similar magnitude results in nearly no mean bias in NAEFS. Keep in mind on individual days, the NAEFS ensemble mean may show significant random errors, even with the overall improvement.

Now for the statistical properties of NAEFS versus GEFS and CEFS:

Brier Skill Score or BSS is used to measure the ratio of times a forecast hits its verification against some benchmark, such as climatology.

This graphic shows the fall 2007 BSS for GEFS, CEFS, and NAEFS Northern Hemisphere 500-hPa heights. The BSS is about equal for CEFS and GEFS, while NAEFS has a ½ to 2.5 day or greater increase in skill.

The BSS can be broken down into:

1. Reliability: how well the ensemble probability distributions match verification distributions. For example, if EFS predicts a 40% chance of rain, and rain subsequently occurs about 40% of the time, that EFS exhibits good reliability. Lower values mean better reliability.
2. Resolution: how well EFSs predict climate anomalies. For example, if EFS forecasts of colder than normal weather correspond to colder than normal verifications, those EFSs have good resolution. Higher values mean better resolution.

Where reliability is greater than or equal to resolution, probabilistic forecast skill is lost.

This graphic breaks down fall 2007 BSSs for the EFSs into reliability and resolution. Identity of each EFS is colored as in the graphic legend at upper right. Reliability is better in NAEFS from day 2 on than in GEFS, and from day 4 on in CEFS. GEFS has poorer reliability than the other ensembles beyond day 2, so its forecast probabilities are not as good. Resolution is better in NAEFS than in GEFS or CEFS. GEFS and CEFS perform similarly to day 5; GEFS is somewhat better thereafter. Thus, NAEFS is better at predicting anomalies than GEFS or CEFS alone.

Where reliability and resolution lines cross for each ensemble, there is loss of probabilistic forecast skill. The relatively poor reliability in GEFS hurts its skill level, even though its resolution is somewhat better than CEFS. Colored arrows for each EFS indicate where probabilistic skill is lost. We see that skill is lost in GEFS at about 7.8 days, in CEFS at about 8.2 days, and in NAEFS at about 10 days, or about 2 days later than in either of the models making it up.

To summarize this section:

1. An ensemble forecast system or EFS is a set of NWP forecasts valid at the same time
2. Why use EFSs? Because the atmosphere is non-linear or chaotic, which makes NWP forecasts sensitive both to the initial atmospheric conditions and the architecture of the NWP model, including its physics and dynamics.
3. NAEFS is a multimodel ensemble containing the GEFS and CEFS
4. GEFS is a single model EFS using Ensemble Transform Bred Vectors for its initial conditions
5. CEFS is a multi-model EFS through use of different physical parameterizations, and perturbations of the data going into initial conditions to create initial condition uncertainty
6. NAEFS has better probabilistic and ensemble mean forecasts than GEFS, and equal to or better probabilistic and ensemble mean skill than CEFS.

Ready to look at some specific NAEFS products you can use when going through the forecast process? Then you’re ready to move on to section 2.