Authors: Johannes Breidenbach (*), David Ellison, Hans Petersson, Jonas Fridman, Terje Gobakken, Rasmus Astrup, Erik Næsset
*Correspondence to: job@nibio.no
Keywords: Global Forest Watch, Landsat, Remote Sensing, National Forest Inventory, Greenhouse Gas Inventory
Abstract
We generally appreciate the position of Ceccherini, Duveiller et al. (2022) that ground truth and remotely sensed data need to be combined to achieve reliable estimates. However, Ceccherini, Duveiller et al. (2022) make several incorrect and unfounded claims and accusations about Breidenbach, Ellison et al. (2022). We think these claims should be corrected. We respond to each of three basic claims by Ceccherini, Duveiller et al. (2022). Due to space limitations, we have focused on what we consider the principal points.
Response to claim (1)
“Increased sensitivity of the GFC product after 2015. […] Breidenbach et al. are not bringing any new element to what has been discussed in Nature Matters
Arising (Ceccherini et al. 2021)” (Ceccherini, Duveiller et al. 2022).
First, none of the previous Matters Arising comments (Palahí, Valbuena et al. 2021; Wernick, Ciais et al. 2021) attempted any validation of the Global Forest Change (GFC) data and were thus only able to hypothesize about possible causes of data discrepancies. Our analysis provides a fundamental contribution to the debate because it decisively illustrates, based on ground truth data, that the GFC data output cannot accurately represent change in harvested area and requires extensive validation. In the Ceccherini, Duveiller et al. (2021) Reply to the Matters Arising critiques, a sampling-based approach is used to correct for inconsistencies in GFC data. Their approach, however, is flawed as will be described in the next section.
Second, in an attempt to correct for the erroneous results in Ceccherini, Duveiller et al. (2020), the Breidenbach, Ellison et al. (2022) paper was first submitted as a Matters Arising comment to Nature in January 2021. The code and data used were made public on the 21st of March, 2021 (Breidenbach, Ellison et al. 2021), so that Ceccherini et al., could verify our critique. After months of waiting, the Nature editors elected not to publish our comment. So, we published an open preprint on the 6th of April, 2021 (Breidenbach, Ellison et al. 2021). The Matters Arising Reply (Ceccherini, Duveiller et al. 2021) was published on the 28th of April, 2021, several months after our objections had been made known to Ceccherini et al., and several weeks after our objections had been openly published.
The following claim is therefore incorrect:
“The impact of [… the change in the GFC algorithm] on harvest statistics has been assessed and reported for the first time in our rebuttal and accompanying documents” (Ceccherini, Duveiller et al. 2022).
Response to claim (2)
“Circularity in sample-based validation due to Landsat data. […] The visual assessment of Landsat-derived NDVI time series was used only to attribute to a specific year the harvest [… The] increased sensitivity of the GFC algorithm clearly did not affect the validation” (Ceccherini, Duveiller et al. 2022).
We are well aware that the Landsat data were only used to determine the timing of changes (harvest) in observed aerial images. However, the original Landsat images were not used for this purpose. Instead, Ceccherini, Duveiller et al. (2021) used a Landsat-based image mosaic. Due to dense cloud cover, the actual change (harvest) year can be incorrect, due to interpolation. Mosaic images also frequently contain other adjustments, such as color alignments, etc., that can modify the original data. Ceccherini, Duveiller et al. (2022) claim the GFC algorithm causes this problem and not the data. However, this is just a claim that remains to be demonstrated.
Furthermore, the GFW providers themselves clearly highlighted their concerns about the need for data validation, stating;
“It must also be noted that a full validation of the results incorporating Landsat 8 has not been undertaken. Such an analysis may reveal a more sensitive ability to detect and map forest disturbance with Landsat 8 data. If this is the case then there will be a more fundamental limitation to the consistency of the mapped interannual loss before and after the inclusion of Landsat 8 data, and a validation of Landsat 8-incorporated loss detection is planned.” (UMD 2015).
This statement clearly highlights that there are likely inconsistencies in the data itself, and not (only) in the GFC algorithm. Further, we are convinced it is not good practice to use Landsat in any form to validate a Landsat-based product (such as the GFC data output).
Response to claim (3)
“Inconsistencies between remote sensing and NFIs validation. […]
in the following paragraphs, we demonstrate that these shortcomings [of Breidenbach et al. 2022] are undermining the validity of the validation exercise presented, to the point that the results cannot be considered as a reference for the evaluation of other studies” (Ceccherini, Duveiller et al. 2022).
We disagree with this claim based on answers to specific claims below.
“Notably, also Breidenbach et al. (2022), in the very last paragraph of the Appendix, consider their approach nonoptimal for estimating actual harvested area” (Ceccherini, Duveiller et al. 2022).
The aim of Breidenbach, Ellison et al. (2022) was to understand why Ceccherini, Duveiller et al. (2020) had come up with such erroneous estimates of increases in harvested area in Finland and Sweden. Official harvest statistics already exist in Finland and Sweden. We had no desire to repeat these statistics. Further, demonstrations of how to correctly combine and use NFI and remote sensing data likewise already exist (e.g. McRoberts, Wendt et al. 2002; McRoberts, Vibrans et al. 2016; Breidenbach, Ivanovs et al. 2021). Such methodologies can also easily be applied to the estimation of harvested area.
“Difficulty in assessing commission errors. The analysis presented by Breidenbach et al. (2022), as stated also by the authors, does not allow a full assessment of the commission error on the harvest statistics (areas where GFC assumes a harvest event that is not confirmed)” (Ceccherini, Duveiller et al. 2022).
Ceccherini, Duveiller et al. (2022) seem to believe, the Breidenbach, Ellison et al. (2022) approach would not be able to handle commission error. We refer to the caption of Fig. 2 in Breidenbach, Ellison et al. (2022) where the time series indicated in yellow is “no loss recorded in the field = commission error”. This claim by Ceccherini, Duveiller et al. (2022) is thus clearly wrong.
“Despite [… a] sharp reduction in the sampling during the 2016–2018 period, the uncertainty range for final felling in Fig. 1 in Breidenbach et al. (2022) remains identical to the previous years, while we expect them to increase” (Ceccherini, Duveiller et al. 2022).
This is yet another undocumented claim by (Ceccherini, Duveiller et al. 2022). First, the confidence intervals in Fig. 1 in Breidenbach, Ellison et al. (2022) are clearly larger towards the end of the period. Second, the increase in the confidence interval may be smaller than Ceccherini et al., expect. But, the estimates in Fig. 1 are ratios with an estimated variance provided by an estimator (Equation (9)) in Breidenbach, Ellison et al. (2022). Further, both these estimator equations and the data itself have been available to Ceccherini et al. since March 2021 (as noted above). If anyone can point us to an error in our calculations, we would be happy to provide a correction.
“Difficulty to have an appropriate a priori stratified sampling. The spatial sampling of NFIs based on permanent plots cannot be stratified a priori by stable forest and harvest classes as recommended in sample-based validation schemes (Olofsson et al. 2014; GFOI 2016)” (Ceccherini, Duveiller et al. 2022).
Stratified sampling schemes increase the efficiency of the estimators, since less reference data is needed to obtain adequately precise estimates. However, due to the comparatively large number of observations, NFIs are also capable of yielding adequate precision even though they are not made-for-purpose, as demonstrated, for example in Breidenbach, Ellison et al. (2022). Further, although stratification is often a wise strategy, it is not a requirement for validating GFC or any other remote sensing data output.
“Difficulty in the temporal attribution of loss year.
Given the uncertainty related to the periodic sampling of ground data (5 years interval) combined with that of the GFC classification, the temporal attribution of the management operation is different between NFIs surveys and GFC. For this reason, Breidenbach et al. (2022) forced the loss year of the NFIs plot with GFC data where the latter were available, introducing a considerable uncertainty in the process” (Ceccherini, Duveiller et al. 2022).
If anything, our methodology results in a more favorable assessment of the GFC accuracy by Breidenbach, Ellison et al. (2022) and improves the overall accuracy of NFI assessments. Therefore, we do not agree that we have introduced increased “uncertainty”.
“Second, Breidenbach et al. (2022) recognize that NFIs use “stand-level observations around the sample plots for area estimation rather than only plot-level measurements,” leading to a clear spatial mismatch between satellite retrievals and the ground truth” (Ceccherini, Duveiller et al. 2022).
Ceccherini, Duveiller et al. (2022) have deleted the context for this citation, leaving a false impression about what we intended to say. The complete sentence is the following;
“Second, official NFI statistics include measurements from both permanent and temporary sample plots and utilize stand-level observations around the sample plots for area estimation rather than only plot-level measurements.” (Breidenbach, Ellison et al. 2022)
In other words, the official statistics use stand-level observations because they reach acceptable levels of accuracy and do not require satellite data support. Breidenbach, Ellison et al. (2022) did not compare stand-level observations to GFC. This is a clear misrepresentation of what we have done.
Though more claims in Ceccherini, Duveiller et al. (2022) could easily be rebutted, due to space limitations, we have focused on what we consider the principal points.
References
Breidenbach J, D Ellison, H Petersson, KT Korhonen, HM Henttonen, J Wallerman, J Fridman, T Gobakken, R Astrup, E Næsset (2021) Finnish and Swedish NFI data intersected with GFC. Zenodo Preprint. doi: https://doi.org/10.5281/zenodo.4625358
Breidenbach J, D Ellison, H Petersson, KT Korhonen, HM Henttonen, J Wallerman, J Fridman, T Gobakken, R Astrup, E Næsset (2021) Harvested area did not increase abruptly – How an inconsistency in satellite-based mapping led to erroneous conclusions. Zenodo Preprint. doi: https://doi.org/10.5281/zenodo.4662921
Breidenbach J, D Ellison, H Petersson, KT Korhonen, HM Henttonen, J Wallerman, J Fridman, T Gobakken, R Astrup, E Næsset (2022) Harvested area did not increase abruptly—how advancements in satellite-based mapping led to erroneous conclusions. Annals of Forest Science 79:2. doi: 10.1186/s13595-022-01120-4
Breidenbach J, J Ivanovs, A Kangas, T Nord-Larsen, M Nilsson, R Astrup (2021) Improving living biomass C-stock loss estimates by combining optical satellite, airborne laser scanning, and NFI data. Canadian Journal of Forest Research 51:1472-1485. doi: 10.1139/cjfr-2020-0518
Ceccherini G, G Duveiller, G Grassi, G Lemoine, V Avitabile, R Pilli, A Cescatti (2020) Abrupt increase in harvested forest area over Europe after 2015. Nature 583:72-77
Ceccherini G, G Duveiller, G Grassi, G Lemoine, V Avitabile, R Pilli, A Cescatti (2021) Reply to Wernick, IK et al.; Palahí, M. et al. Nature 592:E18-E23
Ceccherini G, G Duveiller, G Grassi, G Lemoine, V Avitabile, R Pilli, A Cescatti (2022) Potentials and limitations of NFIs and remote sensing in the assessment of harvest rates: a reply to Breidenbach et al. Annals of Forest Science 79:31. doi: 10.1186/s13595-022-01150-y
McRoberts RE, AC Vibrans, C Sannier, E Næsset, MC Hansen, BF Walters, DV Lingner (2016) Methods for evaluating the utilities of local and global maps for increasing the precision of estimates of subtropical forest area. Canadian Journal of Forest Research 46:924-932
McRoberts RE, DG Wendt, MD Nelson, MH Hansen (2002) Using a land cover classification based on satellite imagery to improve the precision of forest inventory area estimates. Remote Sensing of environment 81:36-44
Palahí M, R Valbuena, C Senf, N Acil, TA Pugh, J Sadler, R Seidl, P Potapov, B Gardiner, L Hetemäki (2021) Concerns about reported harvests in European forests. Nature 592:E15-E17
UMD (2015) Global Forest Change 2000–2015 Data Download. University of Maryland. https://earthenginepartners.appspot.com/science-2013-global-forest/download_v1.3.html. Accessed 14 July 2022
Wernick IK, P Ciais, J Fridman, P Högberg, KT Korhonen, A Nordin, PE Kauppi (2021) Quantifying forest change in the European Union. Nature 592:E13-E14
Authors’ affiliations:
Johannes Breidenbach (*), Department of Forestry and Forest Resources, Norwegian Institute of Bioeconomy Research (NIBIO), Ås, Norway
David Ellison, Department of Forest Resource Management, Swedish University of Agricultural Sciences, Umeå, Sweden; Land Systems and Sustainable Land Management Unit, Institute of Geography, University of Bern, Bern, Switzerland; Ellison Consulting, Baar, Switzerland
Hans Petersson, Department of Forest Resource Management, Swedish University of Agricultural Sciences, Umeå, Sweden
Jonas Fridman, Department of Forest Resource Management, Swedish University of Agricultural Sciences, Umeå, Sweden
Terje Gobakken, Faculty of Environmental Sciences and Natural Resource Management, Norwegian University of Life Sciences, Ås, Norway
Rasmus Astrup, Department of Forestry and Forest Resources, Norwegian Institute of Bioeconomy Research (NIBIO), Ås, Norway
Erik Næsset, Faculty of Environmental Sciences and Natural Resource Management, Norwegian University of Life Sciences, Ås, Norway