This module offers a comprehensive description of a set of common verification measures for hydrologic forecasts, both deterministic and probabilistic. Through use of rich illustrations, animations, and interactions, this module explains how these verification measures can provide valuable information to users with varying needs. In addition to providing a measure of how well a forecast matches observations, verification measures can be used to help forecasters and users learn about the strengths and weaknesses of a forecast.
Have an account?
Sign In to write a review.
- Explain key reasons for performing hydrologic verification and explain important concepts and terminology.
- Define key motivations and purposes of performing forecast verification.
- Explain the need for multiple measures, and why the concept of a “good” forecast is problematic.
- Define and explain the concepts of deterministic and probabilistic forecasts.
- Recognize the seven topics for hydrologic verification and their associated verification measures as created by the NWS Verification Systems Requirements Team.
- Describe key concepts related to the distribution properties of forecasts and observations.
- Define and apply concepts of mean, variance, and standard deviation.
- Describe the purpose of Cumulative Distribution Functions and Probability Density Functions and interpret them.
- Describe the use of the Interquartile Range (IQR) and explain its appropriateness for hydrologic forecast verification.
- Describe the application of a rank histogram to provide useful information about ensemble forecasts.
- Describe key concepts related to measures of forecast confidence.
- Describe how sample size relates to forecast confidence.
- Explain information provided by the confidence interval.
- Define the relationship between confidence level and confidence interval.
- Describe how the confidence interval might be used for statistics.
- Describe key concepts related to correlation measures for forecasts and observations.
- Define correlation.
- Interpret scatter plots to obtain information on correlation.
- Explain use of correlation coefficients.
- Describe key concepts related to categorical forecasts.
- Explain how categorical forecast verification can be applied to either deterministic or probabilistic forecasts.
- Construct and interpret a simple contingency table.
- Describe the meaning of verification statistics that are computed from a contingency table.
- Describe how more complex contingency tables can be used to compute statistics for forecasts associated with more than two categories.
- Explain under what circumstances one would use a Brier Score versus the Ranked Probability Score.
- Explain application of a Brier Score.
- Describe purpose of a Ranked Probability Score.
- Interpret the plot of a Ranked Probability Score.
- Describe key concepts related to error statistics
- Describe use of the Continuous Ranked Probability Score (CRPS).
- Describe the appropriate uses for Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Error (ME).
- Define forecast bias and explain how it is measured.
- Describe key concepts related to forecast skill and how it provides additional information over error statistics
- Describe the general formulation of a skill score.
- Demonstrate the computation of skill scores associated with Root Mean Squared Error, Brier Score, and Ranked Probability Score.
- Describe key concepts related to conditional verification measures.
- Explain the difference between forecast reliability and forecast discrimination.
- Describe how reliability measures and diagrams can help with verifying conditional forecasts.
- Explain what reliability diagrams show.
- Define forecast sharpness.
- Explain what an attributes diagram shows.
- Interpret a discrimination diagram.
- Interpret a Relative Operating Characteristic (ROC) diagram.
hydrology, forecast verification, hydrologic forecasting, hydrologic forecast verification, verification of hydrologic forecasts, flood prediction, flood stage, flood flow, low flow, forecast uncertainty, distribution properties, mean, variance, standard deviation, probabilistic density function, probability density function, PDF, cumulative distribution function, CDF, interquartile range, IQR, box plots, box and whiskers plot, rank histogram, Talagrand diagram, measures of forecast confidence, sample size, confidence interval, correlation of forecasts/observations, scatter plots, correlation coefficient, categorical forecasts, deterministic forecasts, probabilistic forecasts, contingency table, probability of detection, POD, false alarm ratio, FAR, probability of false detection, POFD, false alarm rate, critical success index, CSI, threat score, Brier score, ranked probability score, RPS, continuous ranked probability score, CRPS, error statistics, mean error, mean absolute error, MAE, root mean square error, RMSE, bias, forecast skill, skill score, reference forecast, climatology, persistence, RMSE skill score, Brier skill score, ranked probability skill score, conditional forecast verification, forecast reliability, forecast discrimination, forecast reliability measures, reliability diagram, forecast sharpness, attributes diagram, discrimination diagram, relative operating characteristic, ROC, statistics
