Confidence in Scores
It may be important to know how much confidence to have in the verification scores that you are using. Here we show the confidence interval of the MCRPS scores from the Williamsburg point. Once again we show the trend with lead time for both "all data" together, and just the top 20% of flows.
Since the sample size is smaller for the top 20% of flows than it is for "all flows", there is a larger confidence interval for the high flows as seen with the taller box plots. What these plots tell us is that although the MCRPS may be computed as a value within the box, there is a range that the value can actually be. The smaller boxes for the "all data" set indicated we are more confident that the value is within a smaller possible range of values.
Here is the confidence interval plot for the MCRPS reliability component for the downstream point at Newport. As we showed earlier, there is greater forecast reliability (lower values) indicated in the 48-96 hour lead times for the top 20% of flows. The boxes are also a little more compact at those times, indicating greater confidence that the values fall within a relatively narrow range. Notice that the high flow confidence intervals are never as compact as those for all flows (red box plots).
