Scientific studies and mathematical models are increasingly used to guide the management and development of society. But while science and modelling can indeed provide a robust basis for decision making, we must be mindful of two related considerations. First, science is based not on trust but on skepticism, meticulous technique and careful verification. Second, science is not made of absolute truths, but of hypotheses supported by various degrees of evidence. The degree of confidence we can place in scientific results strongly depends on the methods used to generate them: the more robust and stringent are the methods, the more credible and reliable are the results.
However, the presentation of scientific research is seeing an ever increasing emphasis on the “results”, often at the expense of the “methods” used to generate them. For example, many scientific journals treat methods as second-class citizens by presenting them in a reduced font size, or relegating them to appendices or supplementary materials. Perhaps there is an implicit trust that the methods are “correct”. Yet such trust is risky. While the results are perhaps the most exciting aspects of scientific research, minimized “methods” sections can lead to overlooked flaws in the methodology, and eventually to poor-quality or incorrect results being disseminated. The risk is particularly high when the understanding of reality is entrusted to highly abstract mathematical models, which are used to simulate variables that go far beyond those that can be measured. In this case, apparently routine methodological choices can significantly influence the conclusions of case studies and investigations. Therefore, it is important for the scientific community to resist the temptation to focus on results at the expense of methods, and to challenge the results with stringent verification procedures.
Good advice is of little use if not accompanied by good examples.
Our commentary, Fenicia and Kavetski (2021), recently published in HPToday, comes from hydrology and reflects on the vagaries of publishing the findings of a modelling study on the characterization of hydrological processes in a Luxembourgish catchment. The study goes through several rounds of reviews, which through increasingly stringent and inquisitive verification techniques change the perception of reality, to the point of overturning the initial results.
The characters of this story are the authors, whose stoic resistance to the adversities of the review process is recognized, the reviewers, who ultimately are the gate keepers of science, and the editor, who has the difficult task of mediating between authors and reviewers.
Figure reproduced from Fenicia and Kavetski (2021): a) comparison with a null hypothesis to “weigh” the merits of the proposed model; b) space-time validation of model simulations, and c) controlled stepwise comparisons to appraise the effect of individual model choices.
The commentary highlights the following methodological tools, shown schematically in the above Figure: (i) benchmarking model results with a null-hypothesis model, (ii) testing model predictions in space-time validation, and (iii) carrying out controlled model comparisons. These methodological tools, arguably still underutilized by the hydrological community, offer important diagnostic capabilities to increase the rigor of hydrological and environmental model applications.
Therefore their more widespread application – and keeping methods as first class citizens in scientific publications – is encouraged.
Edited by B. Schaefli
