Effective Use of NWP in the Forecast Process: Introduction

Mr. LeRoy Spayd, Chief
NWS Training Division

Dr. Bill Bua, Project Scientist
UCAR COMET Program

Welcome to the new COMET® Numerical Weather Prediction Training Series entitled "Effective Use of NWP in the Forecast Process". My name is LeRoy Spayd, and I am the Chief of the National Weather Service Training Division. I used to work forecast shifts at NCEP’s Hydrometeorological Prediction Center.

As my career has progressed, I have seen the value of training and continuing education, resulting in forecasters making better warning decisions and better forecasts for the National Digital Forecast Database. During these past 30 years the numerical models have improved dramatically and developing cyclones which were difficult to accurately predict two days ahead of time now are sometimes indicated at Day 6 or 7. 

But our NWS forecasters have continued to add value over the numerical guidance due to continued training and education. For example, here we have monthly threat scores for WRF-NMM, GFS, and human (HPC) forecasts for day 1, 24-hour precipitation exceeding 1” over the continental US for August 2008 through August 2009. WRF-NMM is blue, GFS is brown, and human forecasts are purple. We see that the human beats both NWP models throughout the period. The same results are obtained for day 2. 

Also, model resolution has decreased from the synoptic scale to now resolving convective systems with amazingly realistic-looking detail. However much work remains to accurately model the convective timing, position, and precipitation amounts, as you'll note in these graphics.

First we have the base reflectivity from the national radar composite for 2058 UTC on 22 September 2009. We now superimpose the simulated reflectivity forecast from a 4-km version of WRF-NMM from 12 UTC on 22 September 2009 valid at the same time. There is much realistic-looking detail, including cellular convection in many areas; unfortunately, some of that convection is either misplaced on nonexistent.

The NWP Training Series will help you pull useful information out of such high-resolution models without being misled by realistic but incorrect details in them.

My job as Chief of the NWS Training Division is to help you grow as a forecaster so you can perform your job better. This helps the NWS accomplish its mission of protecting lives and property and helps the American public make better informed weather decisions.

Humans will continue to play an essential role in the forecast process, a role which demands situational awareness and requires understanding to interpret, select, blend, modify, and sometimes discard numerical weather prediction model output and statistical guidance.

To help you perform these tasks, to keep up with the changing models, and to learn about emerging new models and model output, COMET has created a new Numerical Weather Prediction Training Series. Some sections additionally feature short cases for you to run on your AWIPS Weather Event Simulator, also known as WES.

For this series to be a success in improving your office's warnings and digital forecasts, a commitment is needed at all levels. Your commitment of time and effort is needed, not merely going through the motions to pass a quiz, but learning something that helps you become a better forecaster and applying it in your daily work. A commitment from the management in the forecast office is needed to foster a culture of learning and to make sure your schedule allows you the time for training.

Before turning this over to your guide for the remainder of the introduction, I would like to thank the numerous NWS employees in the regional and weather forecast offices who generously gave their time to shape the series structure and serve as subject matter experts, bringing the field perspective into the training. This series would not have been possible without their dedication.

Also, I would like to thank the Warning Decision Training Branch for expertly integrating the series principles into the WES cases.

Finally, the NWS would like to thank UCAR's COMET Program for finalizing the content in this series. COMET's award winning instructional and graphics design staff makes the learning experience enjoyable.

Now it is time to introduce Bill Bua, a member of the COMET NWP training team, who will be your guide for the remainder of this orientation. Bill will start off with a question to help you to reflect on what the role of NWP in the forecast process should be.

Thank you, Leroy for the introduction, and for your support of the NWS employees who contributed their time and expertise to the development of this training series. In this orientation, we will

1. Demonstrate that humans not only can but do add value to NWP forecasts, even as the models are improving,
2. State the overarching series goals and philosophy
3. Provide an overview of the series content
    

Ultimately, this series should help you better understand some of the roles of the human in the forecast process – so you can better add value to NWP model guidance. Prerequisite to that is having a sense of the role of NWP in the forecast process, so let’s start there.

The correct answer is (c). NWP is intended to give guidance to the human weather forecaster. NWP is neither perfect nor a replacement for the human forecaster.

Answer (a) is incorrect because meteorological details of mesoscale convective features are typically not accurately placed in a forecast; even in high-resolution models that explicitly predict convection.

Answer (b) is incorrect because each forecast is giving a possible forecast solution; there is no a priori way of knowing which forecast cycle is more likely to verify.

Answer (d) makes two assumptions – that there is a model of the day and that you can determine ahead of time which model it is. Frequently, different aspects of the forecast are handled better by different models, in which case there isn’t really a model of the day. Other times, after eliminating extremely unlikely outlier solutions and model runs showing key errors in initial conditions, a variety of different model forecasts will remain, all having similar skill, and it may not be possible to identify which is better ahead of time. As a forecaster, you are the decision maker in the forecast process, while models and downstream products such as MOS provide you with guidance which requires your evaluation. We’ll demonstrate how this evaluation process improves the NWP forecast with some verification data on the next slide.

Some people envision calibrated, bias-corrected ensembles and high-resolution models feeding an automated forecast system which could practically replace human forecasters.

As Mr. Spayd showed us at the outset, however, human forecasters add value to precipitation forecasts. And in case you were wondering if that was only true for the 13 months ending in August 2009, here’s another graphic of Eta or WRF-NMM, GFS, and HPC forecast threat scores, but this time, annual scores for the 16 calendar years ending in 2008 (1997 data is missing for the GFS). The same color scheme is used as before.

Notice the general rising tide of overall forecast improvement during the 16 year period. It’s clear that human forecast skill is tied to NWP models; 84 percent of the variance in human forecaster skill during this period is statistically explained by the variance in model skill.

Most importantly, notice that the human forecaster has consistently performed better than the NWP models every year and the amount of improvement, or value added, over NWP has remained pretty much the same through the entire 16-year period, with percentage improvement over NWP skill peaking in 2008 at around 30%. We can conclude that human forecasters add value on average, but the model baseline from which they begin is important as well.