Techniques in Hydrologic Forecast Verification

• 1.0 Introduction
  • Objectives
  • Recommended Prerequisite
  • RFC Case Study Examples
  • IVP and EVS Case Studies
  • IVP: Deterministic Approach
  • EVS: Probabilistic Approach
• 2.0 IVP Case Study: OHRFC
  • Basins Studied
  • Forecast Scenarios
  • Basin Aggregation Assessment
    • Slow-response Basin Aggregation
    • Aggregation Outlier
    • Medium-response Basin Aggregation
    • Fast-response Basin Aggregation
    • Compare Fast- and Slow-response Bias
  • Forecast Verification: Errors and Bias
    • MAE for Forecast Scenarios
    • ME (bias) for Forecast Scenarios
  • Correlation and Lead Time
    • Correlation Over All Lead Times
    • Correlation: Short vs. Long Lead Time
    • Correlation Plots
  • Forecast Skill and Lead Time
  • QPF Influence
    • NPVU
    • QPF Impact with IVP
  • Summary of OHRFC Case
  • Review Questions
• 3.0 EVS Case Study: MARFC
  • Basin and Data Characteristics
    • The Data
    • Sample Sizes
  • Verification Scores
    • Data Distribution
    • Mean Continuous Ranked Probability Score (MCRPS)
    • MCRPSS (Skill Score)
    • Conditional Forecast Verification
    • Confidence in Verification
  • Examining Ensemble Means
    • RMSE at Headwater
    • RMSE of QPF Input
    • Mean Error (Bias) of QPF input
    • Mean Error (Bias) of Flow Forecasts
    • Correlation of QPF Input
    • Correlation for Flow Forecasts
  • Ensemble Spread: Box Plots
    • EVS Box Plots
    • Headwater Box Plot: 6-Hour Lead
    • Headwater Box Plot: 30-Hour Lead
    • Headwater Box Plot, Low Flows, Hour 30
    • Headwater Box Plot Questions
    • Headwater Box Plot: 78 hour Lead
    • Headwater Box Plot: 126 Hour Lead
    • Headwater Box Plot Zoomed: 126 Hour Lead
    • Downstream vs. Headwater: 30 Hour Lead
    • Downstream vs. Headwater � 78 Hour Lead
  • QPF Box Plots
    • QPF Box Plots � 24 Hour Lead
    • Climatologically-based QPF Box Plots
  • Mean Continuous Ranked Probability Score (MCRPS)
    • MCRPS of Flow Forecasts
  • Mean Continuous Ranked Probability Skill Score (MCRPSS)
    • Skill of Flow Forecasts
    • Skill with Respect to Lead Time
  • Forecast Reliability: MCRPS
    • Headwater Forecast Reliability
    • Downstream Forecast Reliability
    • QPF Reliability
  • Forecast Reliability: Spread-Bias Diagram
    • Overforecasting on the Spread-bias Diagram
    • Observations Outside the Forecast Distribution
    • Underforecasting on the Spread-bias Diagram
    • Spread-bias Diagram: Lead Hour 6
    • Spread-bias Diagram: Lead Hour 30
    • Spread-bias Diagram: Lead Hour 78
    • Spread-bias Diagram: Lead Hour 126
  • Forecast Discrimination: ROC
    • ROC Curve for QPF
    • ROC Curve for Climatologically-based QPF
    • ROC Score: QPF
    • ROC Scores: Flow Forecasts
  • Confidence in Scores
  • Review Questions
• 4.0 Verification Techniques Summary
  • Meaningful Verification
  • Interpreting Results: OHRFC
  • Interpreting Results: MARFC
  • Summary and Resources

Skill of Flow Forecasts

Here we see the skill scores for the downstream point at Newport.

Note the decrease in skill score with lead time when all flows are taken together (red) or just the high flows (blue). There is a leveling out in the 36-84 hour period that is more obvious in the high flows plot. This is showing the impact of QPF. The QPF has a positive impact on skill. At longer lead times the impact of the QPF decreases, and the skill scores fall to near zero, showing little skill over climatology.