Lessons from a comprehensive validation of an agent based-model

The experience of the Pampas Model of Argentinean agricultural systems

Federico E. Bert, Santiago L. Rovere, Charles M. Macal, Michael J. North, Guillermo P Podesta

Research output: Contribution to journalArticle

24 Citations (Scopus)

Abstract

There are few published examples of comprehensively validated large-scale land-use agent-based models (ABMs). We present guidelines for doing so, and provide an example in the context of the Pampas Model (PM), an ABM aimed to explore the dynamics of structural and land use changes in the agricultural systems of the Argentine Pampas. Many complementary strategies are proposed for validation of ABM's. We adopted a validation framework that relies on two main streams: (a) validation of model processes and components during model development, which involved a literature survey, design based on similar models, involvement of stakeholders, and focused test scenarios and (b) empirical validation, which involved comparisons of model outputs from multiple realistic simulations against real world data. The design process ensured a realistic model ontology and representative behavioral rules. As result, we obtained reasonable outcomes from a set of initial and simplified scenarios: the PM successfully reproduced the direction of the primary observed structural and land tenure patterns, even before calibration. The empirical validation process lead to tuning and further development of the PM. After this, the PM was able to reproduce not only the direction but also the magnitude of the observed changes. The main lesson from our validation process is the need for multiple validation strategies, including empirical validation. Approaches intended to validate model processes and components may lead to structurally realistic models. However, some kind of subsequent empirical validation is needed to assess the model's ability to reproduce observed results.

Original languageEnglish (US)
Pages (from-to)284-298
Number of pages15
JournalEcological Modelling
Volume273
DOIs
StatePublished - Feb 10 2014

Fingerprint

farming system
survey design
land tenure
land use change
stakeholder
calibration
land use

Keywords

  • Agent-based model
  • Agriculture
  • Land tenure
  • Land use
  • Validation

ASJC Scopus subject areas

  • Ecological Modeling

Cite this

Lessons from a comprehensive validation of an agent based-model : The experience of the Pampas Model of Argentinean agricultural systems. / Bert, Federico E.; Rovere, Santiago L.; Macal, Charles M.; North, Michael J.; Podesta, Guillermo P.

In: Ecological Modelling, Vol. 273, 10.02.2014, p. 284-298.

Research output: Contribution to journalArticle

Bert, Federico E. ; Rovere, Santiago L. ; Macal, Charles M. ; North, Michael J. ; Podesta, Guillermo P. / Lessons from a comprehensive validation of an agent based-model : The experience of the Pampas Model of Argentinean agricultural systems. In: Ecological Modelling. 2014 ; Vol. 273. pp. 284-298.
@article{60456a7d49b64ed1ac67899b05015ae7,
title = "Lessons from a comprehensive validation of an agent based-model: The experience of the Pampas Model of Argentinean agricultural systems",
abstract = "There are few published examples of comprehensively validated large-scale land-use agent-based models (ABMs). We present guidelines for doing so, and provide an example in the context of the Pampas Model (PM), an ABM aimed to explore the dynamics of structural and land use changes in the agricultural systems of the Argentine Pampas. Many complementary strategies are proposed for validation of ABM's. We adopted a validation framework that relies on two main streams: (a) validation of model processes and components during model development, which involved a literature survey, design based on similar models, involvement of stakeholders, and focused test scenarios and (b) empirical validation, which involved comparisons of model outputs from multiple realistic simulations against real world data. The design process ensured a realistic model ontology and representative behavioral rules. As result, we obtained reasonable outcomes from a set of initial and simplified scenarios: the PM successfully reproduced the direction of the primary observed structural and land tenure patterns, even before calibration. The empirical validation process lead to tuning and further development of the PM. After this, the PM was able to reproduce not only the direction but also the magnitude of the observed changes. The main lesson from our validation process is the need for multiple validation strategies, including empirical validation. Approaches intended to validate model processes and components may lead to structurally realistic models. However, some kind of subsequent empirical validation is needed to assess the model's ability to reproduce observed results.",
keywords = "Agent-based model, Agriculture, Land tenure, Land use, Validation",
author = "Bert, {Federico E.} and Rovere, {Santiago L.} and Macal, {Charles M.} and North, {Michael J.} and Podesta, {Guillermo P}",
year = "2014",
month = "2",
day = "10",
doi = "10.1016/j.ecolmodel.2013.11.024",
language = "English (US)",
volume = "273",
pages = "284--298",
journal = "Ecological Modelling",
issn = "0304-3800",
publisher = "Elsevier",

}

TY - JOUR

T1 - Lessons from a comprehensive validation of an agent based-model

T2 - The experience of the Pampas Model of Argentinean agricultural systems

AU - Bert, Federico E.

AU - Rovere, Santiago L.

AU - Macal, Charles M.

AU - North, Michael J.

AU - Podesta, Guillermo P

PY - 2014/2/10

Y1 - 2014/2/10

N2 - There are few published examples of comprehensively validated large-scale land-use agent-based models (ABMs). We present guidelines for doing so, and provide an example in the context of the Pampas Model (PM), an ABM aimed to explore the dynamics of structural and land use changes in the agricultural systems of the Argentine Pampas. Many complementary strategies are proposed for validation of ABM's. We adopted a validation framework that relies on two main streams: (a) validation of model processes and components during model development, which involved a literature survey, design based on similar models, involvement of stakeholders, and focused test scenarios and (b) empirical validation, which involved comparisons of model outputs from multiple realistic simulations against real world data. The design process ensured a realistic model ontology and representative behavioral rules. As result, we obtained reasonable outcomes from a set of initial and simplified scenarios: the PM successfully reproduced the direction of the primary observed structural and land tenure patterns, even before calibration. The empirical validation process lead to tuning and further development of the PM. After this, the PM was able to reproduce not only the direction but also the magnitude of the observed changes. The main lesson from our validation process is the need for multiple validation strategies, including empirical validation. Approaches intended to validate model processes and components may lead to structurally realistic models. However, some kind of subsequent empirical validation is needed to assess the model's ability to reproduce observed results.

AB - There are few published examples of comprehensively validated large-scale land-use agent-based models (ABMs). We present guidelines for doing so, and provide an example in the context of the Pampas Model (PM), an ABM aimed to explore the dynamics of structural and land use changes in the agricultural systems of the Argentine Pampas. Many complementary strategies are proposed for validation of ABM's. We adopted a validation framework that relies on two main streams: (a) validation of model processes and components during model development, which involved a literature survey, design based on similar models, involvement of stakeholders, and focused test scenarios and (b) empirical validation, which involved comparisons of model outputs from multiple realistic simulations against real world data. The design process ensured a realistic model ontology and representative behavioral rules. As result, we obtained reasonable outcomes from a set of initial and simplified scenarios: the PM successfully reproduced the direction of the primary observed structural and land tenure patterns, even before calibration. The empirical validation process lead to tuning and further development of the PM. After this, the PM was able to reproduce not only the direction but also the magnitude of the observed changes. The main lesson from our validation process is the need for multiple validation strategies, including empirical validation. Approaches intended to validate model processes and components may lead to structurally realistic models. However, some kind of subsequent empirical validation is needed to assess the model's ability to reproduce observed results.

KW - Agent-based model

KW - Agriculture

KW - Land tenure

KW - Land use

KW - Validation

UR - http://www.scopus.com/inward/record.url?scp=84890349350&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84890349350&partnerID=8YFLogxK

U2 - 10.1016/j.ecolmodel.2013.11.024

DO - 10.1016/j.ecolmodel.2013.11.024

M3 - Article

VL - 273

SP - 284

EP - 298

JO - Ecological Modelling

JF - Ecological Modelling

SN - 0304-3800

ER -