Model Uncertainty

[Dean]

One major criticism I keep hearing from people is that SLAMM does not model any uncertainty in its forecasts. Of course, one can spatially show some uncertainty by modeling various SLR scenarios (e.g. 1m, A1B, etc) and showing a range of options. However, is there a way to show an "envelope" or range of possibilities around one particular scenario (such as 1m)?

I have seen uncertainty modeled for bathtub type models in a recent publication, assessed against Geodetic benchmarks.

I was wondering what the effect of running the 1m scenario with the MTL changed to both MHW and MLW (and correspondingly changing the MHW and MLW by equal amoints in similar directions) for independent runs would do? Would this work, and potentially show a range of possibilities or uncertainty in the model?

Any feedback on these thoughts from folks working with SLAMM, especially regarding how to show model uncertainty would be great!

Thanks,
Dean

[Jonathan S. Clough]

We are in the process of adding a stochastic uncertainty analysis element to SLAMM with funding from Ducks Unlimited.  This will allow for Monte Carlo analysis to help assess parametric uncertainty.  This can already be done with third-party software and the command-line option as shown in this paper:

http://cedb.asce.org/cgi/WWWdisplay.cgi?264843

Model uncertainty is certainly a complex topic and includes structural model uncertainty, parametric model uncertainty, and observed data uncertainty (which must be teased out from variability).  The EPA guidance on Monte Carlo analysis is a useful document...

http://www.epa.gov/oswer/riskassessment/rags3adt/pdf/rags3adt_complete.pdf

Uncertainty on elevation data is especially tricky and important.  It would be incorrect to modify  the entire elevation dataset by the RMSE or the 5th to 95th percentile for the elevation data set.  That is because those statistics refer to a single lidar return.  Usually a SLAMM cell is comprised of the average of many LiDAR returns so the error would be less even on an individual cell basis.  Furthermore, to assume that all cells would simultaneously be subject to the 5th percentile worst error would require a probability of 0.05 raised to the number of cells.  i.e. a very small probability or impossibility.  For this reason a more sophisticated error analysis that takes into account error on a cell by cell basis with spatial autocorrelation is required. 

It is also incorrect to suggest that you cannot run SLR analysis of less than 0.6 meters using LiDAR basis for this same reason.  The RMSE is not a measure of elevation bias, but of individual LiDAR return accuracy.

Hope this is useful and sorry about the delay in response.  -- J