Techniques in Hydrologic Forecast Verification

• 1.0 Introduction
  • Objectives
  • Recommended Prerequisite
  • RFC Case Study Examples
  • IVP and EVS Case Studies
  • IVP: Deterministic Approach
  • EVS: Probabilistic Approach
• 2.0 IVP Case Study: OHRFC
  • Basins Studied
  • Forecast Scenarios
  • Basin Aggregation Assessment
    • Slow-response Basin Aggregation
    • Aggregation Outlier
    • Medium-response Basin Aggregation
    • Fast-response Basin Aggregation
    • Compare Fast- and Slow-response Bias
  • Forecast Verification: Errors and Bias
    • MAE for Forecast Scenarios
    • ME (bias) for Forecast Scenarios
  • Correlation and Lead Time
    • Correlation Over All Lead Times
    • Correlation: Short vs. Long Lead Time
    • Correlation Plots
  • Forecast Skill and Lead Time
  • QPF Influence
    • NPVU
    • QPF Impact with IVP
  • Summary of OHRFC Case
  • Review Questions
• 3.0 EVS Case Study: MARFC
  • Basin and Data Characteristics
    • The Data
    • Sample Sizes
  • Verification Scores
    • Data Distribution
    • Mean Continuous Ranked Probability Score (MCRPS)
    • MCRPSS (Skill Score)
    • Conditional Forecast Verification
    • Confidence in Verification
  • Examining Ensemble Means
    • RMSE at Headwater
    • RMSE of QPF Input
    • Mean Error (Bias) of QPF input
    • Mean Error (Bias) of Flow Forecasts
    • Correlation of QPF Input
    • Correlation for Flow Forecasts
  • Ensemble Spread: Box Plots
    • EVS Box Plots
    • Headwater Box Plot: 6-Hour Lead
    • Headwater Box Plot: 30-Hour Lead
    • Headwater Box Plot, Low Flows, Hour 30
    • Headwater Box Plot Questions
    • Headwater Box Plot: 78 hour Lead
    • Headwater Box Plot: 126 Hour Lead
    • Headwater Box Plot Zoomed: 126 Hour Lead
    • Downstream vs. Headwater: 30 Hour Lead
    • Downstream vs. Headwater � 78 Hour Lead
  • QPF Box Plots
    • QPF Box Plots � 24 Hour Lead
    • Climatologically-based QPF Box Plots
  • Mean Continuous Ranked Probability Score (MCRPS)
    • MCRPS of Flow Forecasts
  • Mean Continuous Ranked Probability Skill Score (MCRPSS)
    • Skill of Flow Forecasts
    • Skill with Respect to Lead Time
  • Forecast Reliability: MCRPS
    • Headwater Forecast Reliability
    • Downstream Forecast Reliability
    • QPF Reliability
  • Forecast Reliability: Spread-Bias Diagram
    • Overforecasting on the Spread-bias Diagram
    • Observations Outside the Forecast Distribution
    • Underforecasting on the Spread-bias Diagram
    • Spread-bias Diagram: Lead Hour 6
    • Spread-bias Diagram: Lead Hour 30
    • Spread-bias Diagram: Lead Hour 78
    • Spread-bias Diagram: Lead Hour 126
  • Forecast Discrimination: ROC
    • ROC Curve for QPF
    • ROC Curve for Climatologically-based QPF
    • ROC Score: QPF
    • ROC Scores: Flow Forecasts
  • Confidence in Scores
  • Review Questions
• 4.0 Verification Techniques Summary
  • Meaningful Verification
  • Interpreting Results: OHRFC
  • Interpreting Results: MARFC
  • Summary and Resources

Mean Continuous Ranked Probability Score (MCRPS)

Because we are working with probabilistic forecasts, we will look at measures of probabilistic errors using the Mean Continuous Ranked Probability Score (MCRPS). In particular, we will look at the behavior of the MCRPS with lead time to understand the impact of ensemble QPF input.