Best practices for calibrating ecosystem models for managing marine food webs emerge in EcoScope study

A study by EU-funded EcoScope project researchers has produced a series of best practice recommendations to evaluate the appropriateness of vulnerability multipliers and their impact on model uncertainty, specifically intended to improve the calibration of ecosystem-based models used in managing marine food web systems.

“We dived into the theory behind different calibration approaches and looked at how they impact model outcomes. We used this information to put forward a series of best practice recommendations which we hope will increase our ability to understand and communicate model assumptions and limitations,” explained Jacob Bentley, who, together with David Chagaris, Marta Coll, Johanna J Heymans, Natalia Serpetti, Carl J Walters and Villy Christensen, authored the paper, titled “Calibrating ecosystem models to support ecosystem-based management of marine systems”, which was published January 24, 2024 in Oxford’s ICES Journal of Marine Science.

“We hope that this article can increase the transparency of the calibration approach and the impact it can have in these complex ecosystem models, to help support their operational use in ecosystem-based management”, said Bentley in a tweet and video on X.

In the study, the researchers explored existing approaches for calibrating Ecopath with Ecosim (EwE), which Bentley noted is “the world’s most widely used food we modelling tool”.

The researchers formulated the following recommendations, aimed primarily at improving the evaluation of the appropriateness of vulnerability multipliers and their impact on model uncertainty:

• Limit the number of vulnerability multipliers to be estimated - Estimations per predator are the most efficient; adding predator-prey combinations should only be done when demonstrably necessary. Estimating vulnerabilities for groups for which time series are not available should also be avoided, as the estimates produced without sufficient parameters are often flawed and unrealistic.

• Explain the vulnerability multipliers applied in the model, whether newly formulated or using the default multipliers provided. Do they make sense in the context of the particular ecology of the predator and predator-prey interaction, for instance?

• Sense check carrying capacities. It is important to review how predator biomass responds to changes in mortality rates (reduced fishing or predation, for instance) or in prey biomass, and critically evaluate whether the changes modelled are in fact plausible, or if upper and lower limits should be set on such estimates.

• Look beyond goodness of fit when evaluating model performance - Combinations of vulnerability multipliers that ostensibly achieve the best statistical fit do not necessarily produce the “best” model, particularly if other model outputs (such as indicators, reference points and forward projections) appear to be unlikely, the researchers warn. A wider assessment of the model’s performance should therefore be undertaken to review the vulnerability multipliers.

• Perform vulnerability multiplier sensitivity analyses, in order to acknowledge and communicate uncertainty.  

“As our marine environment changes and decision-makers pursue their commitment to the ecosystem-based management and ecosystem-based fisheries management, we've seen a rise in the need and requests for the use of complex ecosystem modelling tools in an operational capacity,” Bentley commented.

“Complex questions about the marine environment, such as how multiple species, pressures and ecosystem services all interact together often lead to complex answers. This means we need to be comfortable communicating our advice and evidence in terms of uncertainty, risk and tradeoffs,” he added.

The research was based on two case studies showing how calibration approaches can impact model-derived advice. The first case study explores how vulnerability multipliers emerge based on the chosen calibration approach using simulated data, while the second takes two operational EwE models (Irish Sea and Northwest Atlantic Continental Shelf) and compares their outputs when calibrated following alternate calibration approaches.