A framework for benchmarking land models

Y. Q. Luo, J. T. Randerson, G. Abramowitz, C. Bacour, E. Blyth, N. Carvalhais, P. Ciais, D. Dalmonech, J. B. Fisher, R. Fisher, P. Friedlingstein, K. Hibbard, F. Hoffman, Deborah N Huntzinger, C. D. Jones, C. Koven, D. Lawrence, D. J. Li, M. Mahecha, S. L. NiuR. Norby, S. L. Piao, X. Qi, P. Peylin, I. C. Prentice, W. Riley, M. Reichstein, Christopher R Schwalm, Y. P. Wang, J. Y. Xia, S. Zaehle, X. H. Zhou

Research output: Contribution to journalArticle

161 Citations (Scopus)

Abstract

Land models, which have been developed by the modeling community in the past few decades to predict future states of ecosystems and climate, have to be critically evaluated for their performance skills of simulating ecosystem responses and feedback to climate change. Benchmarking is an emerging procedure to measure performance of models against a set of defined standards. This paper proposes a benchmarking framework for evaluation of land model performances and, meanwhile, highlights major challenges at this infant stage of benchmark analysis. The framework includes (1) targeted aspects of model performance to be evaluated, (2) a set of benchmarks as defined references to test model performance, (3) metrics to measure and compare performance skills among models so as to identify model strengths and deficiencies, and (4) model improvement. Land models are required to simulate exchange of water, energy, carbon and sometimes other trace gases between the atmosphere and land surface, and should be evaluated for their simulations of biophysical processes, biogeochemical cycles, and vegetation dynamics in response to climate change across broad temporal and spatial scales. Thus, one major challenge is to select and define a limited number of benchmarks to effectively evaluate land model performance. The second challenge is to develop metrics of measuring mismatches between models and benchmarks. The metrics may include (1) a priori thresholds of acceptable model performance and (2) a scoring system to combine data-model mismatches for various processes at different temporal and spatial scales. The benchmark analyses should identify clues of weak model performance to guide future development, thus enabling improved predictions of future states of ecosystems and climate. The near-future research effort should be on development of a set of widely acceptable benchmarks that can be used to objectively, effectively, and reliably evaluate fundamental properties of land models to improve their prediction performance skills.

Original languageEnglish (US)
Pages (from-to)3857-3874
Number of pages18
JournalBiogeosciences
Volume9
Issue number10
DOIs
StatePublished - 2012

Fingerprint

benchmarking
land
ecosystems
climate change
land evaluation
ecosystem response
climate
ecosystem
biogeochemical cycle
vegetation dynamics
prediction
trace gas
model test
land surface

ASJC Scopus subject areas

  • Earth-Surface Processes
  • Ecology, Evolution, Behavior and Systematics

Cite this

Luo, Y. Q., Randerson, J. T., Abramowitz, G., Bacour, C., Blyth, E., Carvalhais, N., ... Zhou, X. H. (2012). A framework for benchmarking land models. Biogeosciences, 9(10), 3857-3874. https://doi.org/10.5194/bg-9-3857-2012

A framework for benchmarking land models. / Luo, Y. Q.; Randerson, J. T.; Abramowitz, G.; Bacour, C.; Blyth, E.; Carvalhais, N.; Ciais, P.; Dalmonech, D.; Fisher, J. B.; Fisher, R.; Friedlingstein, P.; Hibbard, K.; Hoffman, F.; Huntzinger, Deborah N; Jones, C. D.; Koven, C.; Lawrence, D.; Li, D. J.; Mahecha, M.; Niu, S. L.; Norby, R.; Piao, S. L.; Qi, X.; Peylin, P.; Prentice, I. C.; Riley, W.; Reichstein, M.; Schwalm, Christopher R; Wang, Y. P.; Xia, J. Y.; Zaehle, S.; Zhou, X. H.

In: Biogeosciences, Vol. 9, No. 10, 2012, p. 3857-3874.

Research output: Contribution to journalArticle

Luo, YQ, Randerson, JT, Abramowitz, G, Bacour, C, Blyth, E, Carvalhais, N, Ciais, P, Dalmonech, D, Fisher, JB, Fisher, R, Friedlingstein, P, Hibbard, K, Hoffman, F, Huntzinger, DN, Jones, CD, Koven, C, Lawrence, D, Li, DJ, Mahecha, M, Niu, SL, Norby, R, Piao, SL, Qi, X, Peylin, P, Prentice, IC, Riley, W, Reichstein, M, Schwalm, CR, Wang, YP, Xia, JY, Zaehle, S & Zhou, XH 2012, 'A framework for benchmarking land models', Biogeosciences, vol. 9, no. 10, pp. 3857-3874. https://doi.org/10.5194/bg-9-3857-2012
Luo YQ, Randerson JT, Abramowitz G, Bacour C, Blyth E, Carvalhais N et al. A framework for benchmarking land models. Biogeosciences. 2012;9(10):3857-3874. https://doi.org/10.5194/bg-9-3857-2012
Luo, Y. Q. ; Randerson, J. T. ; Abramowitz, G. ; Bacour, C. ; Blyth, E. ; Carvalhais, N. ; Ciais, P. ; Dalmonech, D. ; Fisher, J. B. ; Fisher, R. ; Friedlingstein, P. ; Hibbard, K. ; Hoffman, F. ; Huntzinger, Deborah N ; Jones, C. D. ; Koven, C. ; Lawrence, D. ; Li, D. J. ; Mahecha, M. ; Niu, S. L. ; Norby, R. ; Piao, S. L. ; Qi, X. ; Peylin, P. ; Prentice, I. C. ; Riley, W. ; Reichstein, M. ; Schwalm, Christopher R ; Wang, Y. P. ; Xia, J. Y. ; Zaehle, S. ; Zhou, X. H. / A framework for benchmarking land models. In: Biogeosciences. 2012 ; Vol. 9, No. 10. pp. 3857-3874.
@article{b9ff047a4c5b432fad28bb2d70435ad2,
title = "A framework for benchmarking land models",
abstract = "Land models, which have been developed by the modeling community in the past few decades to predict future states of ecosystems and climate, have to be critically evaluated for their performance skills of simulating ecosystem responses and feedback to climate change. Benchmarking is an emerging procedure to measure performance of models against a set of defined standards. This paper proposes a benchmarking framework for evaluation of land model performances and, meanwhile, highlights major challenges at this infant stage of benchmark analysis. The framework includes (1) targeted aspects of model performance to be evaluated, (2) a set of benchmarks as defined references to test model performance, (3) metrics to measure and compare performance skills among models so as to identify model strengths and deficiencies, and (4) model improvement. Land models are required to simulate exchange of water, energy, carbon and sometimes other trace gases between the atmosphere and land surface, and should be evaluated for their simulations of biophysical processes, biogeochemical cycles, and vegetation dynamics in response to climate change across broad temporal and spatial scales. Thus, one major challenge is to select and define a limited number of benchmarks to effectively evaluate land model performance. The second challenge is to develop metrics of measuring mismatches between models and benchmarks. The metrics may include (1) a priori thresholds of acceptable model performance and (2) a scoring system to combine data-model mismatches for various processes at different temporal and spatial scales. The benchmark analyses should identify clues of weak model performance to guide future development, thus enabling improved predictions of future states of ecosystems and climate. The near-future research effort should be on development of a set of widely acceptable benchmarks that can be used to objectively, effectively, and reliably evaluate fundamental properties of land models to improve their prediction performance skills.",
author = "Luo, {Y. Q.} and Randerson, {J. T.} and G. Abramowitz and C. Bacour and E. Blyth and N. Carvalhais and P. Ciais and D. Dalmonech and Fisher, {J. B.} and R. Fisher and P. Friedlingstein and K. Hibbard and F. Hoffman and Huntzinger, {Deborah N} and Jones, {C. D.} and C. Koven and D. Lawrence and Li, {D. J.} and M. Mahecha and Niu, {S. L.} and R. Norby and Piao, {S. L.} and X. Qi and P. Peylin and Prentice, {I. C.} and W. Riley and M. Reichstein and Schwalm, {Christopher R} and Wang, {Y. P.} and Xia, {J. Y.} and S. Zaehle and Zhou, {X. H.}",
year = "2012",
doi = "10.5194/bg-9-3857-2012",
language = "English (US)",
volume = "9",
pages = "3857--3874",
journal = "Biogeosciences",
issn = "1726-4170",
publisher = "European Geosciences Union",
number = "10",

}

TY - JOUR

T1 - A framework for benchmarking land models

AU - Luo, Y. Q.

AU - Randerson, J. T.

AU - Abramowitz, G.

AU - Bacour, C.

AU - Blyth, E.

AU - Carvalhais, N.

AU - Ciais, P.

AU - Dalmonech, D.

AU - Fisher, J. B.

AU - Fisher, R.

AU - Friedlingstein, P.

AU - Hibbard, K.

AU - Hoffman, F.

AU - Huntzinger, Deborah N

AU - Jones, C. D.

AU - Koven, C.

AU - Lawrence, D.

AU - Li, D. J.

AU - Mahecha, M.

AU - Niu, S. L.

AU - Norby, R.

AU - Piao, S. L.

AU - Qi, X.

AU - Peylin, P.

AU - Prentice, I. C.

AU - Riley, W.

AU - Reichstein, M.

AU - Schwalm, Christopher R

AU - Wang, Y. P.

AU - Xia, J. Y.

AU - Zaehle, S.

AU - Zhou, X. H.

PY - 2012

Y1 - 2012

N2 - Land models, which have been developed by the modeling community in the past few decades to predict future states of ecosystems and climate, have to be critically evaluated for their performance skills of simulating ecosystem responses and feedback to climate change. Benchmarking is an emerging procedure to measure performance of models against a set of defined standards. This paper proposes a benchmarking framework for evaluation of land model performances and, meanwhile, highlights major challenges at this infant stage of benchmark analysis. The framework includes (1) targeted aspects of model performance to be evaluated, (2) a set of benchmarks as defined references to test model performance, (3) metrics to measure and compare performance skills among models so as to identify model strengths and deficiencies, and (4) model improvement. Land models are required to simulate exchange of water, energy, carbon and sometimes other trace gases between the atmosphere and land surface, and should be evaluated for their simulations of biophysical processes, biogeochemical cycles, and vegetation dynamics in response to climate change across broad temporal and spatial scales. Thus, one major challenge is to select and define a limited number of benchmarks to effectively evaluate land model performance. The second challenge is to develop metrics of measuring mismatches between models and benchmarks. The metrics may include (1) a priori thresholds of acceptable model performance and (2) a scoring system to combine data-model mismatches for various processes at different temporal and spatial scales. The benchmark analyses should identify clues of weak model performance to guide future development, thus enabling improved predictions of future states of ecosystems and climate. The near-future research effort should be on development of a set of widely acceptable benchmarks that can be used to objectively, effectively, and reliably evaluate fundamental properties of land models to improve their prediction performance skills.

AB - Land models, which have been developed by the modeling community in the past few decades to predict future states of ecosystems and climate, have to be critically evaluated for their performance skills of simulating ecosystem responses and feedback to climate change. Benchmarking is an emerging procedure to measure performance of models against a set of defined standards. This paper proposes a benchmarking framework for evaluation of land model performances and, meanwhile, highlights major challenges at this infant stage of benchmark analysis. The framework includes (1) targeted aspects of model performance to be evaluated, (2) a set of benchmarks as defined references to test model performance, (3) metrics to measure and compare performance skills among models so as to identify model strengths and deficiencies, and (4) model improvement. Land models are required to simulate exchange of water, energy, carbon and sometimes other trace gases between the atmosphere and land surface, and should be evaluated for their simulations of biophysical processes, biogeochemical cycles, and vegetation dynamics in response to climate change across broad temporal and spatial scales. Thus, one major challenge is to select and define a limited number of benchmarks to effectively evaluate land model performance. The second challenge is to develop metrics of measuring mismatches between models and benchmarks. The metrics may include (1) a priori thresholds of acceptable model performance and (2) a scoring system to combine data-model mismatches for various processes at different temporal and spatial scales. The benchmark analyses should identify clues of weak model performance to guide future development, thus enabling improved predictions of future states of ecosystems and climate. The near-future research effort should be on development of a set of widely acceptable benchmarks that can be used to objectively, effectively, and reliably evaluate fundamental properties of land models to improve their prediction performance skills.

UR - http://www.scopus.com/inward/record.url?scp=84867437707&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867437707&partnerID=8YFLogxK

U2 - 10.5194/bg-9-3857-2012

DO - 10.5194/bg-9-3857-2012

M3 - Article

AN - SCOPUS:84867437707

VL - 9

SP - 3857

EP - 3874

JO - Biogeosciences

JF - Biogeosciences

SN - 1726-4170

IS - 10

ER -