Abstract
This chapter gives an outline of the activities that need to be done in order to make a model reproducible. Starting with a wholistic view of the whole project lifecycle, aspects such as parameter and output definition, documentation as well as verification and validation are examined. Our goal is to take a step back and look at the variety of tasks that can be done to achieve reproducibility, which is not an end in itself but a step towards achieving credibility of a model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
“Knowledge Building and Interpretation” was added.
- 2.
In reality, a scenario can also contain variations of algorithms, e.g. movement rules.
- 3.
The term “parameter” is used in different meanings across the disciplines. In this chapter, the mathematical/informatical view is presented.
- 4.
Furthermore, a distribution might be discrete or continuous.
- 5.
This task is often supported by mathematical optimisation.
- 6.
A number of aspects described in that report are used as a basis for this chapter.
References
Balci O (1994) Validation, verification, and testing techniques throughout the life cycle of a simulation study. Ann Oper Res 53(1):121–173. DOI 10.1007/BF02136828, URL http://link.springer.com/10.1007/BF02136828
Balci O (1997) Verification, Validation and Accreditation of Simulation Models. In: Proceedings of 1997 Winter Simulation Conference, Atlanta, GA, USA, pp 135–141
Bertini E, Lalanne D (2009) Investigating and reflecting on the integration of automatic data analysis and visualisation in knowledge discovery. SIGKDD Explor 11:9–18
Booch G, Rumbaugh J, Jacobson I (2005) The Unified Modeling Language User Guide, 2nd edn. Object Technology Series. Addison-Wesley Professional
Doran JE, Palmer M (1995) The EOS Project: Integrating Two Models of Palaeolithic Social Change. In: Gilbert N, Conte R (eds) Artificial Societies: The Computer Simulation of Social Life. UCL Press, London, pp 103–125
Freire J, Bonnet P, Shasha D (2012) Computational Reproducibility: State-of-the-Art, Challenges, and Database Research Opportunities. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp 593–596
Grimm V, Berger U, Bastiansen F, Eliassen S, Ginot V, Giske J, Goss-Custard J, Grand T, Heinz SK, Huse G, Huth A, Jepsen JU, Jørgensen C, Mooij WM, Müller B, Pe’er G, Piou C, Railsback SF, Robbins AM, Robbins MM, Rossmanith E, Rüger N, Strand E, Souissi S, Stillman RA, Vabø R, Visser U, DeAngelis DL (2006) A standard protocol for describing individual-based and agent-based models. Ecol Model 198(1–2):115–126
Grimm V, Berger U, DeAngelis DL, Polhill JG, Giske J, Railsback SF (2010) The ODD protocol: A review and first update. Ecol Model 221(23):2760–2768
Hamming R (1962) Numerical Methods for Scientists and Engineers, 1st edn. McGraw-Hill, New York
Healey C, Enns J (2012) Attention and visual memory in visualization and computer graphics. IEEE Trans Visual Comput Graph 18(7):1170–1188
Keim D, Kohlhammer J, Ellis G, Mansmann F (eds) (2010) Mastering the Information Age – Solving Problems with Visual Analytics. Eurographics Association, URL http://www.vismaster.eu/wp-content/uploads/2010/11/VisMaster-book-lowres.pdf
Klügl F (2008) A Validation Methodology for Agent-Based Simulations. In: Proceedings of the 2008 ACM Symposium on Applied Computing, ACM Press, New York, NY, pp 39–43. DOI 10.1145/1363686.1363696, URL http://portal.acm.org/citation.cfm?doid=1363686.1363696
Kornhauser D, Wilensky U, Rand W (2009) Design guidelines for agent based model visualization. J Artif Soc Soc Simulat 12(2):1. URL http://jasss.soc.surrey.ac.uk/12/2/1.html
Law AM, McComas MG (2009) How to Build Valid and Credible Simulation Models. In: Rossetti MD, Hill RR, Johansson B, Dunkin A, Ingalls RG (eds) Proceedings of the 2009 Winter Simulation Conference, Austin, TX, USA, pp 24–33
Louloudi A, Klügl F (2012) Immersive Face Validation: A New Validation Technique for Agent-Based Simulation. In: Proceedings. 2012, IEEE, Wroclaw, Poland, pp 1255–1260
Martin RC (2008) Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall, Upper Saddle Rive
Muaz AN, Hussain A, Kolberg M (2009) Verification & Validation of Agent Based Simulations Using the VOMAS (Virtual Overlay Multi-Agent System) Approach. In: Proceedings of the Second Multi-Agent Logics, Languages, and Organisations Federated Workshops, Torino, Italy
Ogden CK (1940) General Basic English Dictionary. Evans Brothers Limited, London. ISBN: 0874713625
Ogden CK (1968) Basic English: International Second Language. Harcourt, Brace & World, New York. URL http://ogden.basic-english.org/isl.html
Piringer H (2012) A Survey and Assessment of Visualization Approaches in the Context of IFEDH. Tech. Rep., VRVis, URL http://ifedh.dwh.at/sites/default/files/IFEDH_Visualization_Survey.pdf
Polhill JG (2010) ODD updated. J Artif Soc Soc Simulat 13(4):9. URL http://jasss.soc.surrey.ac.uk/13/4/9.html
Popper N (in press) Comparative Modelling & Simulation. PhD Thesis, Vienna University of Technology
Sargent R (2010) Verification and Validation of Simulation Models. In: Proceedings of the 2010 Winter Simulation Conference, Baltimore, MD, pp 166–183
Stodden VC (2010) Reproducible research: addressing the need for data and code sharing in computational science. Comput Sci Eng 12(5):8–12
Tufte E (1983) The Visual Display of Quantitative Information. Graphic Press, Cheshire
Tufte E (1996) Visual Explanation. Graphic Press, Cheschire
Tufte E (2006) Beautiful Evidence. Graphic Press, Cheshire
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Popper, N., Pichler, P. (2015). Reproducibility. In: Wurzer, G., Kowarik, K., Reschreiter, H. (eds) Agent-based Modeling and Simulation in Archaeology. Advances in Geographic Information Science. Springer, Cham. https://doi.org/10.1007/978-3-319-00008-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-00008-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-00007-7
Online ISBN: 978-3-319-00008-4
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)