- Review Paper
Adopting robust decision-making to forest management under climate change
Annals of Forest Science volume 74, Article number: 43 (2017)
Multi-objective robust decision making is a promising decision-making method in forest management under climate change as it adequately considers deep uncertainties and handles the long-term, inflexible, and multi-objective character of decisions. This paper provides guidance for application and recommendation on the design.
Recent studies have promoted the application of robust decision-making approaches to adequately consider deep uncertainties in natural resource management. Yet, applications have until now hardly addressed the forest management context.
This paper seeks to (i) assemble different definitions of uncertainty and draw recommendation to deal with the different levels in decision making, (ii) outline those applications that adequately deal with deep uncertainty, and (iii) systematically review the applications to natural resources management in order to (iv) propose adoption in forest management.
We conducted a systematic literature review of robust decision-making approaches and their applications in natural resource management. Different levels of uncertainty were categorized depending on available knowledge in order to provide recommendations on dealing with deep uncertainty. Robust decision-making approaches and their applications to natural resources management were evaluated based on different analysis steps. A simplified application to a hypothetical tree species selection problem illustrates that distinct robustness formulations may lead to different conclusions. Finally, robust decision-making applications to forest management under climate change uncertainty were evaluated and recommendations drawn.
Deep uncertainty is not adequately considered in the forest management literature. Yet, the comparison of robust decision-making approaches and their applications to natural resource management provide guidance on applying robust decision making in forest management regarding decision contexts, decision variables, robustness metrics, and how uncertainty is depicted.
As forest management is characterized by long decision horizons, inflexible systems, and multiple objectives, and is subject to deeply uncertain climate change, the application of a robust decision-making framework using a global, so-called satisficing robustness metric is recommended. Further recommendations are distinguished depending on the decision context.
The idea of robust, instead of optimal decision making, has recently entered the field of forest management (e.g., Yousefpour and Hanewinkel 2016; McDaniels et al. 2012), and has already established itself as a promising method for dealing with deeply uncertain future conditions, especially in water management. Forest management is characterized by very long-term decision outcomes due to its long production time (Pasalodos-Tato et al. 2013; Schou et al. 2015). For decisions, this implies that they must be suitable not only for present but also for future growth conditions (Schou et al. 2015). Regeneration decisions, for example, are close to irreversible or at least highly inflexible. Yet, future conditions are characterized by various uncertainties (Pasalodos-Tato et al. 2013) that make sound management decisions difficult.
One uncertainty that plays an increasing role in the scientific literature on forest management is climate change uncertainty (e.g., Colloff et al. 2016; Yousefpour and Hanewinkel 2015; Seidl and Lexer 2013). It pertains both to the degree of change and to the environmental response to the changes (Lindner et al. 2014). Future changes in climate might be of such magnitude that are beyond the natural adaptive capacity of forest species or ecosystems, which can lead to local extinctions and the loss of important functions and services (Keenan 2015). Hazards may include drought, pests, and wind damage (Petr et al. 2014) as well as flooding, snow breakage, and fire (Yousefpour et al. 2012). This is why we would like to focus on decision making under climate change uncertainty in particular in this paper.
Most commonly, optimizing methods are used as a decision-support in forest management, which either consider only one future climate scenario or assign a single probability density function (PDF) to future scenarios. Yet climate change uncertainty poses a special challenge because it is classified by many authors (see Sect. 3) as deeply uncertain. Deep uncertainty is characterized by a lack of knowledge or of agreement on a single probability distribution of an uncertain parameter (further explained in Sect. 3). Probabilistic approaches are thus likely to be inadequate for dealing with climate change uncertainty.
In recent years, non-probabilistic, so-called robust approaches and their application in natural resource management (NRM) under climate change uncertainty have attracted research (e.g., Knoke et al. 2016; Daron 2015; Hadka et al. 2015), especially in the field of water management. Robust approaches and their applications are thoroughly discussed in Sects. 4, 5, and 6. Generally, they set themselves apart from optimization approaches by seeking solutions that “satisfice” (adapted from “satisfying” and “sufficing”) performance requirements over a wide range of future scenarios instead of finding a single solution that performs best for a single PDF. Robust approaches are often applied through analytic and highly computational frameworks that have the advantage of exhaustively exploring the decision space. Until now, these frameworks have very sparsely, and not thoroughly, been applied to climate change uncertainty in forest management (e.g., Knoke et al. 2016; McDaniels et al. 2012). Yet, we see a great potential as these frameworks have been successfully applied to handle climate change uncertainty in other fields of NRM.
The aim of this research is thus to assess the potential transferability of robust decision-making frameworks from NRM to forest management decision problems that are affected by deeply uncertain climate change. To achieve this goal, we (i) highlight different levels of uncertainty and existing approaches to deal with each level, (ii) review robust decision-making approaches that particularly deal with deep uncertainties and outline their different frameworks applied to NRM under deep climate change uncertainty, (iii) provide a simple hypothetical forest management decision problem demonstrating the logic behind different robustness metrics and their diverging recommendations, and consecutively (iv) compare approaches to finding robust decisions in forest management with the robust decision-making frameworks applied in NRM. Finally, we conclude and draw recommendations for an application to forest management as well as pointing out how the selection of RDM metrics depends on the attitudes towards risk of decision makers, i.e., degree of the risk aversion and availability of statistical information about uncertain parameters, i.e., their PDFs.
To achieve the aims of this study and the steps outlined above, we compiled an overview of robust decision-making approaches and their application in NRM, including forest management, based on an extensive systematic assessment in the Web of Science database. Different combinations of three keywords (Fig. 1) were used and 122 related studies were found (see Online Resource Table 1). To make sure of capturing all applications of robust decision-making approaches that have possibly already been applied to forest management, we explicitly included forest* as a search term that was used interchangeably with environment* for some search combinations. While some previously known approaches were included in the search by name (Info-Gap Decision Theory (Info-Gap) and robust decision making), this review might have neglected robust decision-making approaches that were not referred to as robust in the title or abstract.
Based on our review results, we categorized uncertainty into different levels and determined adequate methods of integrating each level into decision making. This helped us to pin down deep uncertainty as compared to other levels of uncertainty and to outline adequate approaches on dealing with it in decision making. Out of the approaches we encountered during our research, we picked three frameworks that have been used for dealing specifically with deep uncertainty. We compared them based on the typology introduced by Herman et al. (2015). These three frameworks are robust decision making (RDM), multi-objective robust decision making (MORDM), and Information Gap Decision Theory (Info-Gap). Even though Robust Optimization is frequently applied in NRM under uncertainty, criticism regarding its robustness under deep uncertainty has been raised, as its uncertain values may be based on some form of likelihood or probability distribution (Herman et al. 2015). Therefore, it was not considered for application in forest management, even though it is explained in Sects. 4 and 5 and included in the quantitative literature overview. To demonstrate the impact of robustness metric choice on robustness recommendations, we chose a typical forest management decision problem under climate change uncertainty. The decision problem is hypothetical and numbers were based on experience. Since the aim of this decision problem was to demonstrate the logics and effects of different robustness metrics and not to identify robust solutions to the problem, we chose a limited number of climate scenarios and predefined options. In practice, considering portfolios of tree species instead of single tree species is recommended as diversification is a highly recognized strategy to deal with climate change uncertainty (Knoke et al. 2005). The forest decision problem is defined by a situation in which a forest manager can choose among four tree species for regeneration while being uncertain about the degree of climate change, which is represented by a single climate factor x i instead of typical factors for precipitation and temperature. The higher the climate factor, the higher the one-directional change compared to today’s climate. One species benefits from climate change by an increase in wood production (S Winner1), two are compromised by a decrease in wood production (S Loser1, S Loser2), and one species is unaffected (S Indifferent) by climate change. The performance of each species alternative is measured by the land expectation value (LEV) (Faustmann 1849) in Euro per hectare. Climate change is represented by 10 future scenarios of x i representing a plausible range of climate development. Even though a global sampling of the uncertain parameter(s) is recommended, we use a small amount of scenarios here for computational ease. The LEV of a species depends on the climate indicator x i and two fixed parameters (p, r). The linear equation is given by LEV ij = p j + r j ∗ x i . LEV ij for each species under each climate scenario is given in Online Resource, Table 2. Four robust metrics were applied which correspond to those outlined in Table 1 and will be defined in Sect. 5.4. For metrics M1 and M2, we set a performance requirement of LEV ≥ 470 €/ha. For metrics M2 and M3, we defined the best estimate of climate change as x̂ = 2.5. M2 applies intervals on each side of x̂ in a range [0.5, 5.0] in 0.5 steps (Online Resource, Table 2). The uncertainty horizon (concept explained in Sect. 4) is given by α̂. When using M2, the decision-making process resembles the Info-Gap; for the other three metrics, it resembles the robust decision-making framework.
Finally, methods for finding robust decisions under climate change uncertainty in forest management have been compared to applications of the three analyzed RDM frameworks in NRM under climate change uncertainty based on (i) what decisions are made and how decision alternatives are identified, (ii) which uncertainties are considered, and (iii) how robustness is measured.
For each of these steps, findings are discussed within each paragraph. From all this information, we draw conclusions on adequate robust decision-making methods for dealing with deep uncertainty and the potential of applying these methods to forest management under deep climate change uncertainty, and give recommendations on the design.
Levels of uncertainty
In order to understand for which type of uncertainties robust decision-making approaches are designed, this section reviews different levels of uncertainty and adequate ways for dealing with these.
Generally, uncertainty is a situation in which there is no unique and complete understanding of the system to be managed (Petr et al. 2014), due to a lack of knowledge. Uncertainty can be quantified in various ways, depending on the level of available knowledge.
Figure 2 distinguishes three levels of uncertainty and shows different terms used in the literature for each level. The levels are defined by decreasing knowledge or consequently increasing ignorance and are based on findings from Walker et al. (2003) and Walker et al. (2010). While determinism, i.e., complete knowledge is unattainable, total ignorance/unknown unknowns correspond to situations in which we do not even know what we do not know and which thus are probably impossible to deal with. For descriptions of levels 1–3 uncertainties, refer to Walker et al. (2003) and Walker et al. (2010).
While level 2 implies that probabilities can be assigned to different possible future scenarios, under level 3, there is little scientific basis for placing believable probabilities on scenarios (Walker et al. 2010; Styczynski et al. 2014) and thus non-probabilistic approaches are adequate. Level 3 can also take on an even higher level of ignorance, where scenarios are not known, i.e., we only know that we do not know (Walker et al. 2010) as these are events that lie outside of our expectations due to a lack of experience. They are also referred to as Black Swans (Taleb 2010). From now on, level 3 is referred to as deep uncertainty, as that term is broadly used for this level of uncertainty (Lempert et al. 2003b). Even though levels 1–3 uncertainties are unknowable at present, they may be reduced over time when more and more information and evidence become available. Whether that is possible depends on the nature of an uncertainty. Two important distinctions in environmental system modeling are epistemic and aleatory (Pasalodos-Tato et al. 2013). Epistemic uncertainty arises through the imperfection of our knowledge and can be reduced through additional observations. Aleatory uncertainties represent the randomness of nature but also the unpredictability of social, economic, and cultural dynamics and cannot be reduced through additional observations but can be represented through a frequency distribution (Walker et al. 2003) or stationary statistical variability (Beven 2016) that has an assumed level of certainty. In order to deal and possibly reduce uncertainty, it is important to identify the sources of the uncertainty (Beven 2016). Major sources of uncertainty in model-based decision support are (1) the model structure, (2) the input data, (3) the parameters, and (4) the outcomes of the model, which equals the accumulated uncertainty of 1 to 3 (Maier et al. 2016; Walker et al. 2003). When identifying the sources of uncertainty, reducing the epistemic uncertainty through additional observations can help to move more towards aleatory residual errors.
Climate change uncertainty is characterized as deep for a few reasons. Firstly, many factors influence the magnitude of climatic change, such as future increases in atmospheric greenhouse gases (Keenan 2015), which again largely depend on policies, market, and social factors. Secondly, despite advances in climate modeling, the accuracy of these models to predict future climate change is limited (Mortazavi-Naeini et al. 2015) due to our imperfect knowledge of the functioning of the climate system (epistemic uncertainty), possible occurrence of regime shifts due to positive feedback mechanisms in the climate system (Bonan 2008), natural variability (aleatory uncertainty), and the resulting diverging expert views. Also, climate change uncertainty cannot be observed historically as “future climatic conditions may result in system states that have never previously existed” (Keenan 2015, p. 157) and thus drawing probabilities based on historic data is critical (Schou et al. 2015). These factors inhibit an accurate prediction of the future climate.
Considering all of the above, it can be concluded that deep uncertainty of climate change should be considered in decision-making using a range of plausible scenarios without assigning probabilities or a single probability density function to them. The approaches to deal with level 1 and level 2 are thus inadequate for dealing with level 3 uncertainty. Besides climate change uncertainty, it is important to take into account or at least be aware of all other possible uncertainties and their sources, i.e., where in the model they are located (model structure, input data, or model parameters chosen).
Concepts of robust decision-making
Having defined deep uncertainty and identified non-probabilistic approaches as being adequate to deal with it, this section introduces robust decision-making concepts. The definitions of robustness we are exploring capture the idea that a decision alternative performs satisfactorily over many plausible future scenarios (e.g., Hall et al. 2012; Mortazavi-Naeini et al. 2015) instead of being optimal under one expected scenario. Robustness requires that no probabilities are attached to these future scenarios. Yet, there are different nuances of this main idea, which are summarized in Table 1. First of all, robustness can either be based on a satisficing or regret measure. A decision maker that looks at robustness from satisficing point of view seeks a decision alternative that meets his or her performance requirements over the range of plausible future scenarios. With a regret view, a decision maker wants to minimize the regret of choosing incorrectly, where regret is the loss in performance. This regret could be the cost of assuming the wrong future scenario or the cost of choosing the wrong alternative. In the first case, maximum regret of an alternative is the difference between its performances in the best estimate future scenario and in the scenario where it shows the worst performance, for example, as applied by Kasprzyk et al. (2013). In the second case, the regret of an alternative in a certain future scenario is the difference between its performance and the best-performing alternative, as proposed by Savage (1951). Maximum regret of an alternative is its highest regret achieved over all future scenarios. In both regret cases, the alternative with the smallest maximum regret is the most robust.
A striking difference in the satisficing and regret way to robustness is that satisficing relates robustness to the fulfillment of performance requirements over the range of plausible scenarios, while regret relates an alternative’s performance to either the expected performance or the best alternative. Lempert and Groves (2010) therefore distinguish these two approaches as using absolute performance criteria (satisficing) and relative performance criteria (regret) for identifying robustness. Similar to satisficing, a satisficing-optimizing approach optimizes a constrained objective function. An alternative is robust, if for a set of uncertain parameter values (future scenarios), the constraints (performance requirements) are fulfilled. The alternative that performs best in the worst-case scenario is chosen (based on Wald’s maximin rule, which seeks to minimize the maximum loss; Wald 1939). The set of uncertain parameter values can have different shapes (e.g., box, ellipsoid) (Ben-Tal et al. 2009), which thus also has an effect on what the worst-case scenario is and thus how robustness is defined.
Satisficing-based robustness can further be subdivided into the concepts of global and local satisficing (Hall et al. 2012) (illustrated in Fig. 4). Global satisficing uses a measure similar to the domain criterion (Starr 1963). While the domain criterion quantifies the volume of the uncertainty space in which the decision maker’s performance requirements are met, the global satisficing metric measures the fraction of all plausible future scenarios (result of sampling the plausible ranges of each uncertain factor) that meet the requirements. The higher the fraction, the more robust an alternative is. Yet, a truly robust solution has to fulfill the performance requirements over all plausible future scenarios. In contrast, local satisficing does not take into account the whole uncertainty space but uses the concept of an uncertainty horizon to quantify robustness, an approach developed by Ben-Haim (2006). This approach is embedded in the Info-Gap which is further analyzed in the following section. The uncertainty horizon is the number of deviations from an uncertain parameter’s best estimate that are allowed before the performance requirements are no longer fulfilled (Fig. 4). Local satisficing thus samples outward from a best estimate instead of sampling the whole uncertainty space which is basically the well-established concept of radius of stability (Sniedovich 2012). Local satisficing has been criticized as inadequate under deep uncertainty as under deep uncertainty, a best estimate cannot be made and is likely to prove very wrong (Sniedovich 2012).
Other robustness criteria are Safety First and Limited Degree of Confidence (McInerney et al. 2012). Both balance the goal of maximizing expected utility with minimizing worst-case performance. Limited Degree of Confidence works according to Wald’s maximin principle (Wald 1939) whereas Safety First maximizes expected utility after guaranteeing that utility in the worst-case scenario exceeds a performance requirement. Yet, both criteria partly rely on an expected value and thus include probabilities. They will therefore not be considered further for application under deep uncertainty in this paper.
Robust decision-making frameworks
In order to support exploring the robustness of decision alternatives, different robust decision-making frameworks have been developed. RDM, MORDM, and Info-Gap are the most recognized of them. RDM was developed by Lempert et al. (2003a) as a generic method specifically for conditions of decision-making under deep uncertainty. MORDM combines RDM with many objective evolutionary optimizations and visual analytics, developed by Kasprzyk et al. (2013) for environmental decision-making under deep uncertainty and has been advanced by Hadka et al. (2015) as an open source framework. Info-Gap was developed by Ben-Haim (1985) first for the application to mechanical engineering for unknown spatial distribution and later on put forward as a generic method (Ben-Haim 2006). In the meanwhile, Info-Gap has found a few applications also in environmental management under climate change uncertainty (e.g., Hall et al. 2012; Matrosov et al. 2013).
Robust Optimization (RO) was originally developed by Ben-Tal et al. (2009) as a framework that seeks robust solutions which both remain nearly optimal and satisfy performance constraints across uncertain future scenarios. It has been successfully used for multi-objective decision-making problems under uncertainty in natural resources management (e.g., Gabrel et al. 2014; Knoke et al. 2016; Palma and Nelson 2009). Today, a diversity of RO techniques is applied that differ in approaches to robustness and the quantification of uncertainty. Deb and Gupta (2006), for example, advanced multi-objective RO, which optimizes with multivariate constraints on the deviation of an alternative from an expected performance, which is a probabilistic approach. Due to this diversity, RO’s suitability for deep uncertainty cannot be generalized and we therefore decided to not include it in the following framework comparison.
Criticism has been raised that Info-Gap, similar to RO, is not adequate for use under deep uncertainty as it uses local robustness (as described in Sect. 4) and therefore makes a best estimate of uncertain parameters (Sniedovich 2012). We still integrate it in our framework comparison to highlight the differences and as its robustness metric is often applied within the RDM and MORDM framework in order to compare different views on robustness.
These three frameworks have been extensively described and compared by Herman et al. (2015) based on the basic properties of these frameworks, namely (1) generating decision alternatives, (2) developing states of the world, i.e., future scenarios, (3) measuring robustness of decision alternatives, and (4) identifying which uncertain parameters control performance and how strongly. All three frameworks start with a problem formulation. The sequential analysis steps that the three frameworks follow and which they may approach in different ways are summarized in Fig. 3 and explained in detail below.
Before starting any analysis, all relevant information should be assembled in the decision problem definition. A recommendation is the so-called XLRM-framework (Lempert et al. 2003a), which requires defining (1) the exogenous uncertainties (“X”), i.e., factors that are outside of the control of the decision-maker such as climatic change or its effects on the environment; (2) levers (“L”), i.e., decision variables such as thinning frequency and intensity that together make up a decision alternative; and (3) relationships (“R”), i.e., equations or models that link the decision variables with (4) measures (“M”), i.e., variables that measure the performance of an alternative and are a basis of comparing the desirability of each alternative, e.g., revenue or carbon storage.
Generating decision alternatives
The next step of the decision analysis is identifying alternatives among which the decision makers have to choose and which will be analyzed for robustness. Decision alternatives are combinations of different levels or options of each lever L. They can either be predefined or newly created. RDM and Info-Gap often use predefined alternatives (Herman et al. 2015), while MORDM always creates alternatives through a multi-objective optimization process. Using predefined alternatives may be interesting if the current strategy, or modifications to it, should be evaluated for robustness, which can also be created as a result of an unrobust current strategy (Lempert and Groves 2010). Yet, using a set of alternatives that are state-of-the-art or that have been identified by stakeholders in a previous analysis (Matrosov et al. 2013) may fail in finding robust solutions. Instead, taking into account the whole decision space by generating new alternatives via computational search or strategy tables (McDaniels et al. 2012) increases the chance of finding robust alternatives that may also be a surprise. Computational search may include Latin Hypercube Sampling, a total enumeration of the decision space (all possible combinations of the decision levers (L)) or (multi-objective) optimization (Herman et al. 2015). MORDM performs multi-objective optimization using multi-objective evolutionary algorithms to find a set of solutions that are Pareto-optimal under an expected future scenario (best estimates of the uncertain parameters). Within our reviewed literature, we did not find an application of Info-Gap using multi-objective optimization for generating decision alternatives. Theoretically, such an application would be possible; indirectly, it has already been applied within MORDM by using a robustness metric, which is similar to the Info-Gap approach, on a set of Pareto-optimal alternatives (Hadka et al. 2015).
Generating states of the world
In order to evaluate the performance and robustness of one strategy or several alternatives under uncertain future conditions, the frameworks generate plausible future scenarios from the deeply uncertain parameters, often called states of the world (SOWs). It is important to identify and sample as many uncertain parameters as possible in order to later on proof which of them are influential and which can be neglected. The frameworks use two distinct approaches to generating SOWs: While RDM and MORDM perform a “global” sampling of the uncertain factors over their plausible ranges, Info-Gap samples radially outward from an expected SOW until the predefined performance requirements are not met anymore. Figure 4 demonstrates the difference, where α is the parameter of the deviation step from the expected SOW, and the total number of samples until performance fails is called the uncertainty horizon. All uncertain parameters are sampled outwards simultaneously. RDM and MORDM usually use Latin Hypercube Sampling to create a predefined number of SOWs, generally 1000–10,000.
Having defined the decision alternatives and depicted uncertainty as described above, the robustness of the alternatives to the uncertainties can now be measured. The analyzed frameworks offer four different robustness metrics which mirror the four concepts of robustness: global and local satisficing as well as two regret measures. These metrics have been described in Sect. 4 and in Table 1. The difference between global and local satisficing is in addition illustrated in Fig. 4.
Info-Gap exclusively uses the local satisficing metric. In RDM, the decision makers can decide between using the global satisficing metric and either of the two regret metrics, depending which robustness concept they prefer for a certain context. Lempert and Groves (2010), for example, only apply the global satisficing metric, Hall et al. (2012) exclusively regret to the expected performance. The open-source tool OpenMORDM, which provides all analytical steps and visualizations of MORDM, allows the application of all four robustness metrics, also including Info-Gap’s local satisficing metric. Hadka et al. (2015) demonstrate that comparing all four metrics provides more insights and a more informed way of robust decision making.
As a final step, RDM and MORDM identify ranges of the uncertain values (i.e., scenarios) in which the candidate alternative(s) do not meet performance criteria, defined by the decision makers. The uncertain parameters most responsible for failure, and thus those that most control robustness, are identified. This process helps to create new alternatives that are more robust than those initially considered (Matrosov et al. 2013). Usually, the Patient Rule Induction Method (PRIM) algorithm (e.g., in Lempert and Groves 2010) is applied in RDM to identify vulnerable parameter ranges, i.e., that do not fulfill a predefined performance requirement, but Lempert et al. (2008) have also compared its performance to Classification and Regression Trees (CART), another sensitivity analysis, and found neither approach superior. PRIM visualizes these vulnerable ranges for each parameter, i.e., where an above average number of SOWs are vulnerable. CART, on the other hand, divides the uncertainty space into less and more frequent vulnerabilities but is visually less interpretable than PRIM. While the developers of MORDM (Kasprzyk et al. 2013) only use PRIM, OpenMORDM also offers the option to use CART as they find it complementary. In addition, they include sensitivity analyses that do not identify vulnerable ranges like PRIM and CART, but rank the uncertain factors by their influence on performance metrics (Hadka et al. 2015). These include among others ANOVA and Sobol sensitivity indices. Info-Gap does not apply a sensitivity analysis to the uncertain parameters, because the uncertainty horizon already indicates the magnitude of impact each uncertain parameter has on the failure of an alternative.
A simple application to forest management
Depending on the degree of climate change x i , i.e., the climate change scenario, different tree species have a higher LEV and are thus preferred (Online Resource, Fig. 4). The example shown (see Table 2) illustrates that the different robustness metrics do not necessarily lead to the same robust species decisions. M1 represents RDM’s satisficing-based approach for which only with a performance requirement of LEV ≥4.7 can a robust alternative be found (S indifferent). M2 represents Info-Gap’s local robustness metric and M3 and M4 RDM’s regret metrics with regard to the worst-case and the best alternative, respectively. Robustness does not only depend on the robustness metric, but it can also depend on the choice of performance requirement (metric M1 and M2), expected climate scenario (M2 and M3), plausible range of the climate parameter (M1, M3 and M4), and the set of alternatives (M4). In this example, a slight change in the performance requirement turns S indifferent from the most robust to the least robust choice for the two satisficing metrics M1 or M2 (see Table 2). It is also clear that the optimal species under the expected climate scenario rarely coincide with any robustness recommendation. Overall, it has to be taken into account that this is a very simplistic forest management decision example that does not consider different stakeholder preferences, further objectives, and other uncertain parameters besides climate change that might have an impact on wood production. Also, the option of regenerating a mix of species, as Crowe and Parker (2008) or Knoke et al. (2008), who use a portfolio approach, is not considered.
This section first gives a quantitative overview over the approaches used in NRM for making a robust decision. Next, a selection of case studies is compared that apply one of the above outlined robust decision-making frameworks under deep climate change uncertainty. From these, conclusions are drawn for the application of these frameworks to forest management under climate change uncertainty.
Scenario analysis was the most applied method to find so-called robust solutions under deep uncertainty (Online Resource, Fig. 1). Scenario analysis is not a robust approach by itself. It uses a small number of prespecified scenarios, which often represent a low, high, and no change scenario, instead of creating scenarios over the uncertain parameter ranges like (MO)RDM and Info-Gap. Some papers simply analyzed the variance in performance of an alternative solution between some predefined scenarios (e.g., Robinson et al. 2016) or applied metrics such as minimax (regret) (e.g., Perry 2013 and Prato 2015) or minimum performance requirement, e.g., Ogden and Innes (2007), which resemble the robustness metrics discussed above. Despite the simplicity of scenario analysis compared to RDM, MORDM, and Info-Gap, the limited amount of scenarios cannot adequately cover all interactions between a number of different parameters (Kasprzyk et al. 2013). More case studies explicitly applied at least one of the robust decision-making frameworks RDM, MORDM, and Info-Gap or RO (also in combination with Modern Portfolio Theory). Resilience thinking was also often mentioned as a robust approach to decision making. Yet, it is rather a concept than a framework and has a number of different definitions (Seidl 2014). Out of the robust decision-making frameworks, Info-Gap was applied by far the most within the selection of case studies (Online Resource, Fig. 1), followed by RDM and RO. MORDM and RO, combined with Modern Portfolio Theory, have been barely applied. Yet, a look at the distribution over the years (Online Resource, Fig. 1) shows that Info-Gap has not been applied to NRM so much in recent years, but RDM and even more recently MORDM have experienced an increase.
Most case studies as well as most robust decision-making frameworks dealt with decision problems in water management (see Online Resource, Fig. 2). Forest management was also strongly represented, because we explicitly searched for robust applications in this area alongside natural resource management in general. Yet, applications of robust decision-making frameworks were limited to Info-Gap and a simplified RDM which was applied specifically for decisions on climate change adaptation measures (McDaniels et al. 2012).
Only slightly more than half of the forest management case studies considered climate change uncertainty. Out of these most looked exclusively at climate change as an uncertainty while fewer also took into account other uncertainties (Online Resource, Fig. 3).
Framework applications to climate change uncertainty
This section presents case studies in forest management that analyzed the robustness of decisions under climate change uncertainty (overview in Table 3) in order to identify aspects that should be considered for a potential application of (MO)RDM. Also, lessons that can be learned from the application of (MO)RDM to other contexts in NRM under climate change uncertainty are drawn from a selected number of case studies (listed in Online Resource, Table 3). Even though RDM and MORDM as outlined above have not been applied to forest management decisions yet, some studies have applied what could be called a simplified RDM (McDaniels et al. 2012; Ogden and Innes 2007) or a similar approach (Seidl and Lexer 2013; Seidl et al. 2011) to adaptive forest management under climate uncertainty. McCarthy and Lindenmayer (2007) applied Info-Gap to species selection and Crowe and Parker (2008) used Modern Portfolio Theory to find robust species portfolios. The two simplified RDM mainly set themselves apart from a full RDM approach by (1) using expert judgment instead of simulation models to predict outcomes for each alternative; (2) considering a low, high, and no change scenario of the uncertain climate change parameters instead of a total enumeration of plausible parameter ranges; and (3) not identifying key uncertainties unlike the scenario discovery in (MO)RDM. This makes them less computationally complex and also does not require an extensive amount of data and simulation models which makes them more practicable but possibly less reliable. Nevertheless, using predefined scenarios does not allow identification of threshold values for the deeply uncertain parameters that lead to performance failure of an alternative (Herman et al. 2015). Also, the degree of robustness cannot be determined as precisely as in RDM.
Out of the considered forest management studies, most aimed at finding an entire climate adaptive strategy composed of different decision levers (species composition, harvest, silvicultural measures) which were always discrete variables (prespecified levels or binary (yes/no)). McCarthy and Lindenmayer (2007) and Crowe and Parker (2008), on the other hand, exclusively looked at species choice, the first study as a binary choice between an exotic and a native species and the second as a portfolio of different seed sources. Decision alternatives were either prespecified or newly generated. Similar to Lempert and Groves (2010) who applied RDM in water management, Seidl and Lexer (2013) first analyzed the current strategy for vulnerabilities in order to then deduct adaptive measures to specifically reduce them. Lempert and Groves (2010) additionally allowed for a change in alternatives after the occurrence of a specific signpost. Yet, unlike Lempert and Groves (2010) and the usual RDM process, the adaptive strategies were not tested for robustness. McDaniels et al. (2012), on the other hand, created a set of forest management strategies with a strategy table combining different levels of different silvicultural measures. While a strategy table may be interesting when the decision levers each only have a small amount of discrete levels(e.g., the harvest volume has only three levels) and the number of all possible combinations is small, using multi-objective optimization makes sense especially when continuous levers are used (e.g., a range of possible harvest volumes). Hadka and Reed (2015), for example, have 100 decision levers: the allowable pollution into a lake for every one of the next 100 years, which could be within a certain range and may differ every year. In forest management, this could correspond to deciding on, for example, basal area after thinning every 5 years over the entire rotation period. Similar to Crowe and Parker (2008) who use Modern Portfolio Theory to create portfolios of seed sources that can adapt to climate change by minimizing expected variance and covariance over different climate scenario, species portfolios could also be created using multi-objective optimization and then tested for robustness. This was done by Kasprzyk et al. (2012) who applied MORDM to water management portfolios. For each lever within their portfolio, they indicated a plausible range of values that the lever could have. Correspondingly, species portfolios could contain a set of tree species, the range indicating the proportion of each tree species.
Most of the forest management studies exclusively considered climate change uncertainty, which was represented by the two metrics temperature and precipitation, with the exception of McInerney et al. (2012) who used the mean interval between wildfires as a metric for climate change uncertainty. The uncertain effect, instead of climate change itself, has been considered in robust decision frameworks for example through perturbation of natural hydrology (Matrosov et al. 2015) or water inflow multipliers (Herman et al. 2015). Yet, the results of Seidl and Lexer (2013), who also looked at the effects of social uncertainty (relative importance of sustainable forest management objectives), illustrate the importance of considering other uncertainties that might have an effect on the performance of a strategy. Herman et al. (2015) even found that a socio-economic parameter controlled robustness more than climate change in a water management problem. Socio-economic parameters like demand, prices, and capacity (e.g., Kasprzyk et al. 2013; Herman et al. 2015) might also be important to consider in forest management. Climate change uncertainty was, with exception of Info-Gap, considered through three to five predefined scenarios. Yet, creating scenarios through combining values within the uncertain ranges of each factor has the advantage of a more thorough vulnerability assessment and robustness measure as outlined above.
Measuring the robustness of an alternative first requires measuring its performance under the uncertain scenarios, i.e., how well it fulfills the objectives. Since forest management serves multiple purposes, taking into account different economic, ecological, and social performance metrics should at least be evaluated. The case studies that considered a multi-faceted climate adaptation strategy took into account ecological and socio-economic factors (see Table 3). Seidl and Lexer (2013) and Seidl et al. (2011) subdivided these metrics into those that measure sensitivity to climate change or adaptive capacity. The species choice case studies only had a single performance metric: the net present value of timber yield (McCarthy and Lindenmayer 2007) or the adaptive growth variation between two seed sources (Crowe and Parker 2008). The performance was either measured through ecosystem models or expert judgment (McDaniels et al. 2012; Ogden and Innes 2007). The simulation models need to capture the performance metrics, yet social factors especially may be hard to depict in a model. In which situations expert judgments may be more beneficial than simulation models, especially regarding computational ease and integration of social factors, needs to be explored.
Seidl and Lexer (2013) and Seidl et al. (2011) measure robustness by the variance in performance between three climate change scenarios. Yet, they measure overall performance by weighing the performance metrics which prohibit looking at the trade-offs among them—an important benefit of (MO)RDM. McDaniels et al. (2012), in turn, also look at the performance for each objective in each scenario. Ogden and Innes (2007) come the closest to RDM’s global satisficing metric, since an alternative has to fulfill minimum performance requirements in every scenario in order to be called robust. Yet, in all cases, robustness is measured over three scenarios instead of the typical 10,000. Even though these three scenarios include the low and high end of plausible climate change, they do not measure robustness and vulnerability as precisely as indicated previously. The global satisficing robustness metric was applied in most NRM case studies under climate change uncertainty and is also highly recommended for forest management, since in this way, minimum performance requirements are assured. For example, in forest management, a minimum annual cut in younger forest stands is set to safeguard necessary thinning activities in forest management plans. In particular, in contexts in which failing these requirements may lead to catastrophic situations, e.g., an irreversible lake eutrophication (Lempert and Collins 2007), setting thresholds is crucial.
Conclusions and recommendations
Robust decision-making processes appear to be highly recommendable for forest management decision problems under climate change uncertainty as these are mostly characterized by long decision horizons as well as inflexible systems. Climate change uncertainty has been identified by many studies as a deep uncertainty which should not be treated with a single probability density function but with robust approaches as defined in this study.
We first draw general conclusions on the suitability of the reviewed frameworks and robustness metrics for dealing with deep uncertainty before giving recommendations for an application to forest management.
The review of RDM, MORDM, and Info-Gap has raised the question as to which approaches truly deal with deep uncertainty in an appropriate way and how robustness is captured. In particular, the use of best estimates of the uncertain parameter values seems to be in contradiction to the concept of robustness. Best estimates are used in robust decision-making frameworks in three ways. In MORDM, they are used to find a set of Pareto-approximate solutions. They are also used to measure the regret to the best estimate scenario, and Info-Gap measures robustness outward of a best estimate. Applying best estimates to find a set of candidate alternatives does not directly infringe the concept of robustness, as its purpose is to narrow down the entire decision space to a high-performance and manageable set of alternatives. In case none of the optimal solutions are robust, non-optimal solutions can still be considered for robustness analysis. The application of best estimates for the robustness metrics is more questionable since it directly influences the robustness judgment. Its application under deep uncertainty, especially in Info-Gap, has therefore been criticized in the literature (Singh et al. 2015; Sniedovich 2012; Matrosov et al. 2013; Maier et al. 2016). Thus, using a robustness metric that does not require making best estimates within (MO)RDM may be most suitable under deep uncertainty, as pointed out below.
Based on the insights of this review, Fig. 5 summarizes recommendations for the use of robustness metrics depending on availability of knowledge about probability density functions of uncertain parameters and the risk aversion of decision makers. In situations of little knowledge, i.e. deep uncertainty, only the satisficing and alternative-based regret (MO)RDM are recommendable. A high risk aversion is here equated with basing the decision on the performance in the worst-case scenario or maximizing the number of scenarios in which performance is sufficient. All metrics that are partly based on best estimates suggest that decision makers are less risk-averse than when entirely basing the decision on the worst-case scenario. Info-Gap is only recommendable for decision makers for which lack of knowledge and risk-aversion are both medium, e.g., for timber price uncertainty in forest management.
Decision problems in forest management under climate change uncertainty which could benefit from using (MO)RDM are not only (i) multi-faceted adaptive strategies that comprise a portfolio of measures such as species composition, harvest intensity and rotation length, and fire and pest management but also (ii) only single adaptive measures. RDM is especially suitable if predefined strategies or measures are evaluated. For example, the current management strategy could be evaluated for vulnerabilities. This strategy can then be adjusted to specifically target at reducing these vulnerabilities and thereby increasing robustness. Also, if the decision variables are discrete (e.g., harvest volume: increase, decrease, or no change), alternatives should rather be created through a total enumeration of the decision space rather than using multi-objective optimization. MORDM, on the other hand, is especially suitable if the decision levers are continuous, e.g., harvest volume could take any value within a certain range so that a high-performance selection of alternatives can be generated using the ranges of each decision lever. This does not have to be a strategy composed of different measures but could also be a measure that has to be taken at certain intervals over time, e.g., how much volume is thinned every 10 years over a whole rotation period.
Climate change uncertainties that are relevant to forestry are as follows: the change in annual means and yearly distribution of light, temperature, precipitation, and CO2-content in the atmosphere. This is also an uncertainty in the effects of climate change, e.g., CO2-content on growth (Jacobsen and Thorsen 2003) or the impact on wildfire and pest occurrence. As forest management decisions are also subject to other uncertainties such as timber prices and demand, as well as political decisions, it is crucial to also consider those uncertainties that might have an impact on the performance of a decision and analyze their individual impact on the performance failure of alternatives. Assumptions may be wrong and the impact of some uncertainties under- or overestimated. Next to the uncertainties, the choice of performance and robustness metrics may impact the verdict on how robust an alternative is. Due to the multi-objective nature of forest management, performance should not be measured by only a single metric, but ideally consider both ecological and socio-economical metrics. Examples are given in Table 3. Forest management often sets minimum performance requirements in order to stay sustainable, such as a minimum timber yields or basal area after thinning. In this case, and building on the discussion above on robustness metric choice, a global satisficing robustness metric is an adequate choice as at least a minimum performance is guaranteed over the defined uncertainty space. The current study suggests a potential application of MORDM and RDM to the field of forest management under climate change uncertainty and provides general recommendations for its design. Hence, it offers a basis to further explore an application of these two frameworks in order to test the benefits compared to more traditional decision support methods such as optimization approaches or less computational methods based on expert judgment. Specifically, the characteristics of forest management decision problems have to be further explored, and MORDM or RDM has to be adapted accordingly.
Beh EH, Maier HR, Dandy GC (2015) Scenario driven optimal sequencing under deep uncertainty. Environ Model Softw 68:181–195
Ben-Haim Y (1985) The assay of spatially random material. Reidel, Dordrecht
Ben-Haim Y (2006) Info-gap decision theory. Decisions under severe uncertainty, 2nd edn. Academic Press/Elsevier, Oxford
Ben-Tal A, El Ghaoui L, Nemirovski A (2009) Robust optimization. Princeton University Press, Princeton
Beven K (2016) Facets of uncertainty. Epistemic uncertainty, non-stationarity, likelihood, hypothesis testing, and communication. Hydrol Sci J 61:1652–1665
Bonan GB (2008) Forests and climate change: forcings, feedbacks, and the climate benefits of forests. Science (New York, NY) 320:1444–1449
Colloff MJ, Doherty MD, Lavorel S, Dunlop M, Wise RM, Prober SM (2016) Adaptation services and pathways for the management of temperate montane forests under transformational climate change. Clim Chang:267–282
Crowe KA, Parker WH (2008) Using portfolio theory to guide reforestation and restoration under climate change scenarios. Clim Chang 89:355–370
Daron J (2015) Challenges in using a robust decision making approach to guide climate change adaptation in South Africa. Clim Chang 132:459–473
Deb K, Gupta H (2006) Introducing robustness in multi-objective optimization. Evol Comput 14:463–494
Faustmann M (1849) Calculation of the value which forestland and immature stands possess for forestry. J For Econ 1:7–44
Gabrel V, Murat C, Thiele A (2014) Recent advances in robust optimization. An overview. Eur J Oper Res 235:471–483
Hadka D, Reed P (2015) Large-scale parallelization of the Borg multiobjective evolutionary algorithm to enhance the management of complex environmental systems. Environ Model Softw 69:353–369
Hadka D, Herman J, Reed P, Keller K (2015) An open source framework for many-objective robust decision making. Environ Model Softw 74:114–129
Hall JW, Lempert RJ, Keller K, Hackbarth A, Mijere C, McInerney DJ (2012) Robust climate policies under uncertainty. A comparison of robust decision making and info-gap methods. Risk Anal 32:1657–1672
Hallegatte S (2009) Strategies to adapt to an uncertain climate change. Glob Environ Chang 19:240–247
Herman JD, Zeff HB, Reed PM, Characklis GW (2014) Beyond optimality: Multistakeholder robustness tradeoffs for regional water portfolio planning under deep uncertainty. Water Resour Res 50:7692–7713
Herman JD, Reed PM, Zeff HB, Characklis GW (2015) How should robustness be defined for water systems planning under change? J Water Resour Plan Manag 141:4015012
Jacobsen JB, Thorsen BJ (2003) A Danish example of optimal thinning strategies in mixed-species forest under changing growth conditions caused by climate change. For Ecol Manag 180:375–388
Kasprzyk JR, Reed PM, Characklis GW, Kirsch BR (2012) Many-objective de novo water supply portfolio planning under deep uncertainty. Environ Model Softw 34:87–104
Kasprzyk JR, Nataraj S, Reed PM, Lempert RJ (2013) Many objective robust decision making for complex environmental systems undergoing change. Environ Model Softw 42:55–71
Keenan RJ (2015) Climate change impacts and adaptation in forest management. A review. Ann For Sci 72:145–167
Knight FH (1921) Risk, uncertainty, and profit. Hart, Schaffner and Marx, New York
Knoke T, Stimm B, Ammer C, Moog M (2005) Mixed forests reconsidered: A forest economics contribution on an ecological concept. For Ecol Manag 213:102–116
Knoke T, Hildebrandt P, Klein D, Mujica R, Moog M, Mosandl R (2008) Financial compensation and uncertainty: Using mean-variance rule and stochastic dominance to derive conservation payments for secondary forests. Can J For Res 38:3033–3046
Knoke T, Paul C, Hildebrandt P, Calvas B, Castro LM, Hartl F, Dollerer M, Hamer U, Windhorst D, Wiersma YF, Curatola Fernandez GF, Obermeier WA, Adams J, Breuer L, Mosandl R, Beck E, Weber M, Stimm B, Haber W, Furst C, Bendix J (2016) Compositional diversity of rehabilitated tropical lands supports multiple ecosystem services and buffers uncertainties. Nat Commun 7:11877
Lempert RJ, Collins MT (2007) Managing the risk of uncertain threshold responses: Comparison of robust, optimum, and precautionary approaches. Risk Anal 27:1009–1026
Lempert RJ, Groves DG (2010) Identifying and evaluating robust adaptive policy responses to climate change for water management agencies in the American West. Technol Forecast Soc Chang 77:960–974
Lempert RJ, Popper SW, Bankes SC (2003) Shaping the next one hundred years: New methods for quantitative, long-term policy analysis and bibliography. RAND, Santa Monica
Lempert RJ, Bryant BP, Bankes SC (2008) Comparing algorithms for scenario discovery. RAND, Santa Monica
Lindner M, Fitzgerald JB, Zimmermann NE, Reyer C, Delzon S, van der Maaten E, Schelhaas M-J, Lasch P, Eggers J, van der Maaten-Theunissen M, Suckow F, Psomas A, Poulter B, Hanewinkel M (2014) Climate change and European forests. What do we know, what are the uncertainties, and what are the implications for forest management? J Environ Manag 146:69–83
Maier HR, Guillaume J, van Delden H, Riddell GA, Haasnoot M, Kwakkel JH (2016) An uncertain future, deep uncertainty, scenarios, robustness and adaptation. How do they fit together? Environ Model Softw 81:154–164
Matrosov ES, Woods AM, Harou JJ (2013) Robust decision making and Info-gap decision theory for water resource system planning. J Hydrol 494:43–58
Matrosov ES, Huskova I, Kasprzyk JR, Harou JJ, Lambert C, Reed PM (2015) Many-objective optimization and visual analytics reveal key trade-offs for London’s water supply. J Hydrol 531:1040–1053
McCarthy MA, Lindenmayer DB (2007) Info-gap decision theory for assessing the management of catchments for timber production and urban water supply. Environ Manag 39:553–562
McDaniels T, Mills T, Gregory R, Ohlson D (2012) Using expert judgments to explore robust alternatives for forest management under climate change. Risk Anal 32:2098–2112
McInerney D, Lempert R, Keller K (2012) What are robust strategies in the face of uncertain climate threshold responses? Clim Chang 112:547–568
Mortazavi-Naeini M, Kuczera G, Kiem AS, Cui L, Henley B, Berghout B, Turner E (2015) Robust optimization to secure urban bulk water supply against extreme drought and uncertain climate change. Environ Model Softw 69:437–451
Ogden AE, Innes J (2007) Incorporating climate change adaptation considerations into forest management planning in the boreal forest. Int For Rev 9:713–733
Palma CD, Nelson JD (2009) A robust optimization approach protected harvest scheduling decisions against uncertainty. Can J For Res 39:342–355
Pasalodos-Tato M, Mäkinen A, Garcia-Gonzalo J, Borges JG, Lämås T, Eriksson LO (2013) Assessing uncertainty and risk in forest planning and decision support systems. Review of classical methods and introduction of new approaches. For Syst 22:282
Perry N (2013) The precautionary principle, uncertainty and the Noah’s Ark problem. Wildl Res 40:117
Petr M, Boerboom L, Ray D, van der Veen A (2014) An uncertainty assessment framework for forest planning adaptation to climate change. Forest Policy Econ 41:1–11
Prato T (2015) Conceptual framework for assessing the sustainability of forest fuel reduction treatments and their adaptation to climate change. Sustainability 7:3571–3591
Robinson AP, McLarin M, Moss I (2016) A simple way to incorporate uncertainty and risk into forest harvest scheduling. For Ecol Manag 359:11–18
Savage LJ (1951) The theory of statistical decision. J Am Stat Assoc 46:55
Schou E, Thorsen BJ, Jacobsen JB (2015) Regeneration decisions in forestry under climate change related uncertainties and risks: Effects of three different aspects of uncertainty. Forest Policy Econ 50:11–19
Seidl R (2014) The shape of ecosystem management to come: Anticipating risks and fostering resilience. Bioscience 64:1159–1169
Seidl R, Lexer MJ (2013) Forest management under climatic and social uncertainty: Trade-offs between reducing climate change impacts and fostering adaptive capacity. J Environ Manag 114:461–469
Seidl R, Rammer W, Lexer MJ (2011) Adaptation options to reduce climate change vulnerability of sustainable forest management in the Austrian Alps. Can J For Res 41:694–706
Singh R, Reed PM, Keller K (2015) Many-objective robust decision making for managing an ecosystem with a deeply uncertain threshold response. Ecol Soc 20:1–32
Sniedovich M (2012) Black Swans, New Nostradamuses, Voodoo decision theories, and the science of decision making in the face of severe uncertainty. Intl Trans Op Res 19:253–281
Starr MK (1963) Product design and decision theory. Prentice-Hall, Englewood Cliffs
Styczynski A, Wolf J, Tah S, Bose A (2014) When decision-making processes fail: An argument for robust climate adaptation planning in the face of uncertainty. Environ Syst Decis 34:478–491
Taleb NN (2010) The black swan: The impact of the highly improbable, 2nd edn. Penguin Books, London
Wald A (1939) Contributions to the theory of statistical estimation and testing hypotheses. Ann Math Stat 10:299–326
Walker WE, Harremoes P, Rotmans J, van der Sluijs JP, van Asselt M, Janssen P, Krayer von Krauss MP (2003) Defining uncertainty: A conceptual basis for uncertainty management in model-based decision support. Integr Assess 4:5–17
Walker WE, Marchau VA, Swanson D (2010) Addressing deep uncertainty using adaptive policies: introduction to section 2. Technol Forecast Soc Chang 77:917–923
Yousefpour R, Hanewinkel M (2015) Forestry professionals’ perceptions of climate change, impacts and adaptation strategies for forests in south-west Germany. Clim Chang 130:273–286
Yousefpour R, Hanewinkel M (2016) Climate change and decision-making under uncertainty. Curr Forestry Rep 2:143–149
Yousefpour R, Jacobsen JB, Thorsen BJ, Meilby H, Hanewinkel M, Oehler K (2012) A review of decision-making approaches to handle uncertainty and risk in adaptive forest management under climate change. Ann For Sci 69:1–15
Handling Editor: Barry Alan Gardiner
Contribution of the co-authors
Rasoul Yousefpour: Support in conceptualizing and reviewing the paper.
Roderich von Detten: Support in conceptualizing and reviewing the paper.
Stefan Reifenberg: Support in gathering relevant literature and helping to conceptualize the paper.
Marc Hanewinkel: Support in conceptualizing and reviewing the paper.
Electronic supplementary material
About this article
Cite this article
Radke, N., Yousefpour, R., von Detten, R. et al. Adopting robust decision-making to forest management under climate change. Annals of Forest Science 74, 43 (2017). https://doi.org/10.1007/s13595-017-0641-2
- Deep uncertainty
- Robustness metrics
- Uncertainty levels
- Climate change
- Forest management
- Multi-objective robust decision making