Techniques in Hydrologic Forecast Verification

• 1.0 Introduction
  • Objectives
  • Recommended Prerequisite
  • RFC Case Study Examples
  • IVP and EVS Case Studies
  • IVP: Deterministic Approach
  • EVS: Probabilistic Approach
• 2.0 IVP Case Study: OHRFC
  • Basins Studied
  • Forecast Scenarios
  • Basin Aggregation Assessment
    • Slow-response Basin Aggregation
    • Aggregation Outlier
    • Medium-response Basin Aggregation
    • Fast-response Basin Aggregation
    • Compare Fast- and Slow-response Bias
  • Forecast Verification: Errors and Bias
    • MAE for Forecast Scenarios
    • ME (bias) for Forecast Scenarios
  • Correlation and Lead Time
    • Correlation Over All Lead Times
    • Correlation: Short vs. Long Lead Time
    • Correlation Plots
  • Forecast Skill and Lead Time
  • QPF Influence
    • NPVU
    • QPF Impact with IVP
  • Summary of OHRFC Case
  • Review Questions
• 3.0 EVS Case Study: MARFC
  • Basin and Data Characteristics
    • The Data
    • Sample Sizes
  • Verification Scores
    • Data Distribution
    • Mean Continuous Ranked Probability Score (MCRPS)
    • MCRPSS (Skill Score)
    • Conditional Forecast Verification
    • Confidence in Verification
  • Examining Ensemble Means
    • RMSE at Headwater
    • RMSE of QPF Input
    • Mean Error (Bias) of QPF input
    • Mean Error (Bias) of Flow Forecasts
    • Correlation of QPF Input
    • Correlation for Flow Forecasts
  • Ensemble Spread: Box Plots
    • EVS Box Plots
    • Headwater Box Plot: 6-Hour Lead
    • Headwater Box Plot: 30-Hour Lead
    • Headwater Box Plot, Low Flows, Hour 30
    • Headwater Box Plot Questions
    • Headwater Box Plot: 78 hour Lead
    • Headwater Box Plot: 126 Hour Lead
    • Headwater Box Plot Zoomed: 126 Hour Lead
    • Downstream vs. Headwater: 30 Hour Lead
    • Downstream vs. Headwater � 78 Hour Lead
  • QPF Box Plots
    • QPF Box Plots � 24 Hour Lead
    • Climatologically-based QPF Box Plots
  • Mean Continuous Ranked Probability Score (MCRPS)
    • MCRPS of Flow Forecasts
  • Mean Continuous Ranked Probability Skill Score (MCRPSS)
    • Skill of Flow Forecasts
    • Skill with Respect to Lead Time
  • Forecast Reliability: MCRPS
    • Headwater Forecast Reliability
    • Downstream Forecast Reliability
    • QPF Reliability
  • Forecast Reliability: Spread-Bias Diagram
    • Overforecasting on the Spread-bias Diagram
    • Observations Outside the Forecast Distribution
    • Underforecasting on the Spread-bias Diagram
    • Spread-bias Diagram: Lead Hour 6
    • Spread-bias Diagram: Lead Hour 30
    • Spread-bias Diagram: Lead Hour 78
    • Spread-bias Diagram: Lead Hour 126
  • Forecast Discrimination: ROC
    • ROC Curve for QPF
    • ROC Curve for Climatologically-based QPF
    • ROC Score: QPF
    • ROC Scores: Flow Forecasts
  • Confidence in Scores
  • Review Questions
• 4.0 Verification Techniques Summary
  • Meaningful Verification
  • Interpreting Results: OHRFC
  • Interpreting Results: MARFC
  • Summary and Resources

Objectives

Objectives in this training include demonstrating these important steps in verification:

• Evaluate the important characteristics of the data set,
• Evaluate hydrologic forecast performance,
• Evaluate the impact of Quantitative Precipitation Forecast (QPF) input, and
• Identify and understand sources of error.

We will use a case study approach to explore the use of verification statistics and graphics that provide measures of data distribution as well as quantitative measures of forecast error, bias, correlation, confidence, skill, discrimination and reliability.