Class 4.1

Model analysis: Sensitivity, calibration, validation

See Module 4 in Online Course.

An extensive bibliography based on a compilation by Douglas H. Johnson (Northern Prairie Wildlife Research Center)

Some of the most noteworthy titles are shown in green. I certainly do not expect you to read all this. However it is very desirable that you look at one or more titles to get a better feeling of the problem and how it is discussed in literature.

  • Bart, J. 1995. Acceptance criteria for using individual-based models to make management decisions. Ecological Applications 5:411-420.
  • Beven, K., 1993. Prophecy, reality and uncertainty in distributed hydrological modelling. Advances in Water Resources 16: 41-51.
  • Bunnell, F. L. 1989. Alchemy and uncertainty: what good are models? Gen. Tech. Rep. PNW-GTR-232. Portland, Oregon: USDA, USFS, Pacific Northwest Research Station. 27 pages.
  • Cartwright, N. 1983. How the laws of physics lie, Clarendon Press, Oxford, UK. 153 pages.
  • Caswell, H. 1976. The validation problem. Pages 296-308 in H. H. Shugart and R. V. O'Neill, eds. Systems ecology. Dowden, Hutchinson and Ross, Stroudsburg, Pennsylvania, USA.
  • Caswell, H. 1988. Theory and models in ecology: a different perspective. Ecological Modelling 43:33-44.
  • Cowardin, L. M., and D. H. Johnson. 1979. Mathematics and mallard management. Journal of Wildlife Management 43:18-35.
  • Gauch, H. G., Jr. 1993. Prediction, parsimony and noise. American Scientist 81:468-478.
  • Gentil, S., and G. Blake. 1981. Validation of complex ecosystem models. Ecological Modelling 14:21-38.
  • Gold, H. J. 1977. Mathematical modeling of biological systems--an introductory guidebook. John Wiley and Sons, New York, New York, USA. 357 pages.
  • Goodall, D. W. 1972. Building and testing ecosystem models. Pages 173-194 in J. N. R. Jeffers, ed. Mathematical models in ecology. Blackwell Scientific Publications, Oxford, UK.
  • Grant, W. E. 1986. Systems analysis and simulation in wildlife and fisheries sciences. John Wiley and Sons, New York, New York, USA. 338 pages.
  • Haefner, J. W. 1996. Modeling biological systems: principles and applications. Chapman and Hall, New York, New York, USA. 473 pages.
  • Hamming, R. W. 1975. How do you know the simulation is relevant? Simulation 25:163-167.
  • House, P. W. 1974. Diogenes revisited-the search for a valid model. Simulation 23:117-125.
  • Hurley, J. F. 1986. Summary: development, testing, and application of wildlife-habitat models--the manager's viewpoint. Pages 151-153 in J. Verner, M. L. Morrison, and C. J. Ralph, eds. Wildlife 2000: modeling habitat relationships of terrestrial vertebrates. University of Wisconsin Press, Madison, Wisconsin, USA.
  • Lall, U., D.L. Phillips, K.H. Reckhow, D.P. Loucks, 2002. Quantifying and Communicating Model Uncertainty for Decision Making in the Everglades. Report of the Comprehensive Everglades Restoration PlanÕs Model Uncertainty Workshop, U.S. Army Corps of Engineers, South Florida Water Management District, West Palm Beach, FL. January 15 Š 17, 2002
  • Levins, R. 1966. The strategy of model building in populaiton biology. American Scientist 54:421-431.
  • Leggett, R. W., and L. R. Williams. 1981. A reliability index for models. Ecological Modelling 13:303-312.
  • Mankin, J. B., R. V. O'Neill, H. H. Shugart, and B. W. Rust. 1979. The importance of validation in ecosystem analysis. Pages 309-317 in H. H. Shugart and R. V. O'Neill, eds. Systems ecology. Dowden, Hutchinson and Ross, Stroudsburg, Pennsylvania, USA. Reprint of pages 63-71 in G. S. Innis, ed. 1977. New directions in the analysis of ecological systems, part 1. Simulation Councils, Lajolla, California, USA.
  • Marcot, B. G., M. G. Raphael, and K. H. Berry. 1983. Monitoring wildlife habitat and validation of wildlife-habitat relationships models. North American Wildlife and Natural Resources Conference 48:315-329.
  • Mayer, D. G., and D. G. Butler. 1993. Statistical validation. Ecological Modelling 68:21-32.
  • McCarl, B. A. 1984. Model validation: an overview with some emphasis on risk models. Review of Marketing Agricultural Economics 52:153-173.
  • Miller, D. R. 1974. Model validation through sensitivity analysis. Pages 911- 914 in Summer Computer Simulation Conf.,Houston, Proc., Houston, Tex., Lajolla, Calif.: Simulation Council, Inc. Pages 292-308 in H. H. Shugart, and R. V. O'Neill eds. Systems Ecology. Dowden, Hutchinson and Ross, Inc., Stroudsburg, Pennsylvania, USA.
  • Miller, D. R., G. Butler, and L. Bramall. 1976. Validation of ecological system models. Journal of Environmental Management 4:383-401.
  • Naylor, T. H., J. L. Balintfy, D. S. Burdick, and K. Chu. 1966. Computer simulation techniques. John Wiley and Sons, New York, New York, USA 352 pages.
  • Oreskes, N., K. Shrader-Frechette, and K. Belitz. 1994. Verification, validation and confirmation of numerical models in the earth sciences. Science 263:641-646.
  • Popper, K. 1959. The logic of scientific discovery. Harper and Row, New York, New York, USA.
  • Power, M. 1993. The predictive validation of ecological and environmental models. Ecological Modelling 68:33-50.
  • Reynolds, M. R., Jr., H. E. Burkhart, and R. F. Daniels. 1981. Procedures for statistical validation of stochastic simulation models. Forest Science 27:349- 364.
  • Rice, J. A., and P. A. Cochran. 1984. Independent evaluation of a bioenergetics model for largemouth bass. Ecology 65:732-739.
  • Ripley, B. D. 1987. Stochastic Simulation. Wiley, New York, New York, USA. 237 pages.
  • Rubinstein, M. F. 1975. Patterns of problem solving. Prentice-Hall, Englewood Cliffs, New Jersey, USA. 544 pages.
  • Rykiel, E. J., Jr. 1996. Testing ecological models: the meaning of validation. Ecological Modelling 90:229-244.
  • Schamberger, M. L., and L. J. O'Neil. 1986. Concepts and constraints of habitat-model testing. Pages 5-10 in J. Verner, M. L. Morrison, and C. J. Ralph, eds. Wildlife 2000: modeling habitat relationships of terrestrial vertebrates. University of Wisconsin Press, Madison, Wisconsin, USA.
  • Shugart, H. H., and R. V. O'Neill. 1979. Editors' Comments on Papers 23, 24, and 25. Pages 290-291 in H. H. Shugart, and R. V. O'Neill eds. Systems Ecology. Dowden, Hutchinson and Ross, Inc., Stroudsburg, Pennsylvania, USA.
  • Shrank, W. E., and C. C. Holt. 1967. Critique of "Verification of computer simulation models." Management Science 14:B104-B106.
  • Snee, R. D. 1977. Validation of regression models: methods and examples. Technometrics 19:415-428.
  • Starfield, A. M. 1997. A pragmatic approach to modeling for wildlife management. Journal of Wildlife Management 61:261-270.
  • Starfield, A. M., and A. L. Bleloch. 1986. Building models for conservation and wildlife management. Macmillan Publishing Co., New York, New York, USA. 253 pages.
  • Steinhorst, R. K. 1979. Parameter identifiability, validation, and sensitivity analysis of large system models. Pages 33-58 in G. S. Innis and R. V. O'Neill, eds. Systems analysis of ecosystems. International Co-operative Publishing House. Fairland, Maryland, USA.
  • Stone, M. 1974. Cross-validation choice and assessment of statistical predictions (with discussion). Journal of Royal Statistical Society B 36:111- 147.
  • Usher, M. B. 1971. Developments in the Leslie matrix model. Pages 29-60 in J. N. R. Jeffers, ed. Mathematical Models in Ecology. Blackwell Scientific Publications, Oxford, UK.
  • Van Horn, R. 1969. Validation. Pages 232-251 in T. H. Naylor ed. The design of computer simulation experiments. Duke University Press, Durham, North Carolina, USA.
  • Zeide, B. 1991. Quality as a characteristic of ecological models. Ecological Modelling 55:161-174.

Exercises

Do Exercise 4.1 from the Online Course.
Go here to download a copy of Stella.