Closing the gap between phenotyping and genotyping: review of advanced, image-based phenotyping technologies in forestry

Abstract

Key message

The lack of efficient phenotyping capacities has been recognized as a bottleneck in forestry phenotyping and breeding. Modern phenotyping technologies use systems equipped with various imaging sensors to automatically collect high volume phenotypic data that can be used to assess trees' various attributes.

Context

Efficient phenotyping has the potential to spark a new Green Revolution, and it would provide an opportunity to acquire growth parameters and dissect the genetic bases of quantitative traits. Phenotyping platforms aim to link information from several sources to derive knowledge about trees' attributes.

Aims

Various tree phenotyping techniques were reviewed and analyzed along with their different applications.

Methods

This article presents the definition and characteristics of forest tree phenotyping and reviews newly developed imaging-based practices in forest tree phenotyping.

Results

This review addressed a wide range of forest trees phenotyping applications, including a survey of actual inter- and intra-specific variability, evaluating genotypes and species response to biotic and abiotic stresses, and phenological measurements.

Conclusion

With the support of advanced phenotyping platforms, the efficiency of traits phenotyping in forest tree breeding programs is accelerated.

1 Introduction

Forests are key drivers of terrestrial biodiversity, representing the largest biomass producers. Ensuring that forest productivity meets the needs of a rapidly growing world population is a major challenge (Babin et al. 2021). Forests provide the raw material for many essential human needs, including construction material, paper products, firewood, energy, and many non-timber forest products (e.g., food, oils, and medicinal compounds) (Crist et al. 2017). Additionally, forest trees are often associated with the joint production of forest products and environmental goods, like preserving biodiversity, hunting, carbon storage, mitigating climate change, maintaining water quality, and representing our cultural and patrimonial heritage recreation (Strange et al. 2019).

Due to the recent surge in wood consumption and the need for fast-growing, resilient fiber production plantations, breeding better-adapted trees are imperative. Since whole-genome sequencing has been achieved, plant functional genomics studies have entered the big-data era. However, the acquisition of large-scale phenotypic data has become one of the major bottlenecks hindering plant breeding and functional genomics studies. Nevertheless, recent technological advances provide solutions to relieve this bottleneck and explore advanced methods for large-scale phenotyping data acquisition and processing (Yang et al. 2020; Xie & Yang, 2020). High-quality phenotyping facilitates the selection of superior parents for applied breeding. Accurate plant phenotyping is important for gaining a fundamental understanding of genotype × environment interaction that is critical for successful tree breeding. While tree phenotyping is essential for breeding, it also plays a major role in understanding the extent of genetic diversity within species, populations, and families. It also improves our understanding of intra-specific biodiversity, which might become even more important under the context of climate change and its associated extreme climate events (heat, floods, drought-, frost-stress, etc.).

Plant phenotyping refers to the determination of quantitative or qualitative values for morphological, physiological, biochemical, and performance-related properties, which act as observable proxies between gene(s) expression and environment and are important determinants of growth, quality, and stress resistance characteristics. Forest trees phenotyping is beneficial to varieties development, improving wood quality, matching genotypes to sites and end uses, improving silviculture practices, and selecting the best progenies and parents for advanced generations breeding and selection. Exact trait measurements help in understanding to what extent external factors impact timber yield and forest health and development (Dungey et al. 2018).

Traditionally, plant phenotyping is often performed manually; however, this has become a major bottleneck, as it is laborious, costly, inefficient, prone to errors, poorly adaptable, and in many cases requires destructive sampling (e.g., wood cores, leaf and bud tissue, cuttings). Phenotyping costs can be an issue, especially for traits that are expensive to measure (Lebedev et al. 2020). Recently, the rapid advancements of molecular and genomic high-throughput technologies outpaced the traditional phenotyping methods. It has become clear that phenotyping had constituted a major bottleneck in our ability to capitalize on these advancements. Progress in high throughput phenotyping is therefore urgently needed.

Throughput, i.e., the number of units at the considered organizational level (e.g., canopy, individual plant, leaf, and molecular) that can be measured for a specific trait at a given time, is also an important determinant for phenotyping systems (Dhondt et al. 2013). High-throughput phenotyping (HTP) systems, often defined as being able to image hundreds or thousands of plants per day, are paramount in furthering the understanding of "phenomes" and their genetic underpinning (Fahlgren et al. 2015). With high-throughput technologies, plants can be measured in some cases in a non-destructive fashion, providing useful spatial and temporal information with accuracy and precision far exceeding manual phenotyping. HTP shows great potential for increasing yields through improved forest management and for accelerating genomics-based tree improvement.

With the rapidly increasing sophistication, capability, and miniaturization of imaging sensors, the imaging-based approach is quickly becoming the workhorse strategy for most phenotyping applications. Modern imaging techniques have high resolution and allow for multi-dimensional and multi-parameter data visualization. This ability to perform high-throughput phenotyping through image analysis has increased interest in automated approaches to quantify complex traits with the intent of screening germplasm, improving cultivation, and identifying and quantifying biotic and abiotic stresses. Most tree phenotyping can be acquired by digital imaging sensors and processed by image processing algorithms. With the collection of big phenotypic data of individual trees and populations, especially image data, the development of effective approaches to deal with large-scale image data analysis significantly expands the capability of traditional image processing. This approach allows transferring the knowledge generated from the research to practice.

Here, we reviewed current imaging-based phenotypic engineering efforts to improve tree breeding programs' efficiency (i.e., gain per unit time, cost and effort). We highlighted how imaging technologies actively contribute to acquiring high-dimensional, richly informative datasets about forest trees phenotypes. The most commonly used sensors and platforms for measuring forest trees were summarized, and HTP facilities' application was also described. We discussed the main bottlenecks in phenotyping and the importance of multidisciplinary collaboration between forest geneticists and engineers to overcome this challenge.

2 Characteristic of forest tree phenomics

Tree breeding/improvement is different from most crops breeding programs, as the goal is mainly focused on the gradual moving of the improved population's target trait(s) mean relative to the base population while maintaining adequate diversity for meeting environmental contingencies that might occur during their long rotation span as well as future selection. This goal is fundamentally different from crop plants as the trade-off between gain and diversity is paramount. Tree breeders are thus willingly accepting a reduction in gain for maintaining diversity. Most forest tree species are outbreeding and highly heterozygous, so large amounts of segregating variation should be maintained in the breeding and production populations.

In some cases, multiple trait selection is also practiced, although breeding for growth and yield is generally the most common. Research supporting tree breeding has focused on reducing the long-term protracted breeding cycle, and improving selection efficiency (El-Kassaby and Lstiburek 2009; Grattapaglia 2017). The breeding cycle begins with the phenotypic selection of several hundred superior trees from ecologically defined areas called breeding zone (Lind et al. 2018). Clonal copies of these phenotypically selected trees are grafted and planted in a replicated copied over multiple breeding arboreta to safeguard their genetic legacy, and then crossing and testing are performed to select genetically proven individuals for advanced generation breeding and production populations (White et al. 2007). Marker-based or marker-assisted genomic strategies are being implemented at several stages of the breeding cycle, including selecting superior parents and offspring to overcome the longtime needed for completing a single breeding cycle (Grattapaglia et al. 2018; El-Kassaby et al. 2020; Garrido-Cardenas et al. 2018; Collard et al. 2008; Lema et al. 2018).

Forest trees phenotyping includes assessing attributes affected by genetics, environments, and their interactions to improve our understanding of how they shape the complex traits of trees. Presently, there are many high-throughput phenotyping platforms available worldwide, but the majority are designed for annual field crops (Liu et al. 2019b; Virlet et al. 2017; Bai et al. 2018; Salafian, 2017). The establishment and development of tree phenotyping platforms have not been widely developed owing to trees' characteristics that increase the difficulty of phenotyping.

Firstly, trees reach substantial height, requiring specific phenotyping equipment to reach their tops. The limited space of conventional imaging chambers makes it difficult to measure large trees when they pass certain vegetative growth stages. Controlled environment-based platforms, as they operate in growth chambers or greenhouses, can be used for tree seedling phenotyping; however, phenotypic data collected from mature trees grown in the field are the most valuable.

Secondly, trees have thick branches and dense overlapping canopies, which easily cast shades and cause occlusions among branches and leaves. Thus, it is more challenging to obtain complete and accurate phenotypic information. For example, surveying leaf inclination and leaf angle are more frequent for broad-leaved species than coniferous species. The measurement of leaf angular distribution (LAD) for plants with large and curvy leaves could be very time-consuming, if not impossible (Thapa et al. 2018).

Thirdly, the growth cycle of trees is generally long, and it takes substantially more resources and material to carry out phenotyping than annual crops.

Fourthly, developed trees root system requires higher operating space for phenotyping. Trees root systems are important components representing the metabolic basis of growth and development. The distribution and structure of roots determine trees' ability to utilize soil moisture and nutrients and reflect their level of adaptation to the environment. The main bottleneck in the field is collecting phenotypic data of underground parts by in situ, non-destructive root measurement technologies. Recently, the effectiveness of non-destructive methods in tree root mapping and assessment is demonstrated, particularly that of ground-penetrating radar (Alani, et al. 2020).

3 Forest tree phenotyping captured by imaging techniques

Environmental stresses affect many aspects of tree growth, development, and distribution and can be reflected in phenotypic traits. Rapid, non-invasive, and high-throughput optical sensors and sensing methods have been widely used to elucidate plant phenomics for forests and ecosystems. Plant images and reflectance spectra acquired by optical sensing can uncover the multi-dimensional, multi-environment, and multi-source heterogeneous phenotypic traits of forest trees (Fig. 1) (Zhou, et al. 2020 for subcellular localization; Sun, et al. 2021 for the optical sensing; Pont & Dungey, 2018 for forest stand; Han, 2021 for organ image of Pterocarya stenoptera.). The recent development of sensors and image processing further promotes the use of these technologies, especially for accelerating forest breeding.

Fig. 1

Schematic overview for acquiring phenotypic data by optical sensors for stress responses

A common approach in forest tree breeding is to select the best genotype based on a phenotypic expression under different environmental conditions. Forest tree phenotyping encompasses morphological, physiological, biochemical, and performance traits. These phenotyping traits are increasingly recognized as important for understanding the structure-function relationship in plants, assessing the changing interactions among organisms under climate change (Rewald et al. 2020), sustainable forest management, and tree improvement programs (Benavides et al. 2021).

Trees morphological traits are important for studying forest phenomics and include tree attributes (height, stem form, diameter at breast height (DBH)), leaf traits (leaf area index (LAI), leaf area density (LAD), greenness, color, distribution, and angle), and canopy characteristics (volume, coverage, structure, phenology (e.g., flowering period, leaf coloring time, leaf expansion time, and leaf-fall time)). Trees have to be characterized for their features and attributes related to the population (including the relative position to neighboring trees (social status)). At the same time, physiological trait consists of photosynthetic quality, canopy or leaf temperature, and diseases and pest incidence. In contrast, biochemical traits, for example, comprise chlorophyll and lignin contents and water use efficiency.

Rapidly developing sensor techniques are very helpful for monitoring forest trees phenotypes as each component of tree cells and tissues has wavelength-specific absorbance, reflectance, and transmittance properties. Optical sensors aim to measure a phenotype quantitatively through the interaction between light and trees, such as reflected, absorbed, or transmitted photons (Table 1). Imaging spectroscopy of sensors can provide insight into the drivers of growth dynamics through time (throughout the tree cycle) and space (at the cell/tissue, organ, plant, canopy level) growth patterns and the gathered tree spectroscopy data are useful in quantifying performance, vegetation indices, and resilience to environmental stress (Ge et al. 2019).

Table 1 Imaging techniques for estimating forest trees phenotypic traits

4 Imaging sensors and image analysis used for forest tree HTP

Imaging trees is more than just “taking pictures” as imaging sensors extract growth, yield, and stress features in controlled or field environments and allow real-time monitoring more readily. Collecting plant phenotypic data with sufficient resolution (temporal and spatial) and accuracy represents a challenge in developing standardized methods and protocols for collecting sensor-based data and converting them to “trait data” (Li et al. 2014).

With the continuous advancements in sensing and instrumentation, numerous sensor-based technologies have been developed and applied for monitoring plant growth and performance. The process of capturing, analyzing, and using forest tree phenotyping traits is summarized in Fig. 2.

Fig. 2

The process of capturing, analyzing, and application of forest tree phenotyping traits

Phenotyping platforms use semi- or fully automated facilities, precise environment control, and imaging technologies to comprehensively assess growth, development, performance, and adaptation to stress. Plants can be measured with certain accuracy and precision at different levels of organization, from organs to canopies. Imaging sensors are very helpful for detecting plants’ optical properties, especially for those that cannot be directly seen.

Numerous imaging sensors have been developed to increase the precision, resolution, and throughput, each with its advantages and limitations. Different imaging sensors can be used in forest tree phenotyping depending on the goals of each phenotyping experiment, desired objectives, and outcomes. In contrast to the conventional methods using visual scoring, optical imaging aims for rapid and contact-less measurement of traits in tree morphology and physiology. Table 2 summarizes the most common optical imaging sensors used in plant phenotyping under different environments and the useful information extracted from the images data.

Table 2 Summary of the most common imaging sensors used in plant phenotyping experiments under different application environments

Figure 3 gives an example of Quercus images captured in the LemnaTec3D Scanalyzer system (LemnaTec GmbH, Aachen, Germany) consisting of four imaging chambers: Visible (RGB), steady-state fluorescence, hyperspectral, and thermal infrared. Using a commercial platform and software, such as Python, Matlab, Visual C++, plants features can be extracted from the background for measuring size, color, geometry, and architecture using the visible (RGB) camera. A thermal infrared camera was used to monitor canopy or leaf temperature. Images acquired with a fluorescence camera allowed the assessment of chlorophyll concentration. Reflectance spectra with a hyperspectral camera can estimate leaf water content and canopy health parameters. A robust and accurate method was developed for rapid and noninvasive determination of the phenotypic traits of leaves using visible, steady-state fluorescence, hyperspectral, and thermal infrared images (Zhang et al. 2022). The results suggested that the different imaging systems combined with data fusion could be used synergistically to improve phenotypic traits prediction.

Fig. 3

Illustrative Quercus images captured with the visible, steady-state fluorescence, hyperspectral, and thermal infrared sensors in LemnaTec3D Scanalyzer system. a Quercus image captured with a visible camera. b Quercus image captured with a fluorescence camera. d Quercus image captured with a hyperspectral camera. d Quercus image captured with a thermal infrared camera

Many commercial platforms are developed for a limited range of species, encompassing small plants such as Arabidopsis (Boyes et al. 2001; Goggin et al. 2015) and the primary cereal crops (Bai et al. 2019; Paulus et al. 2014; Bai et al. 2018). The complex characteristics of trees mean higher requirements for platform structure, image processing algorithm, and multi-source sensor integration for phenotypic analysis.

4.1 Visible light imaging

A visible image is based on digital images intended to mimic human perception to provide information or input to systems that need data for plant phenotyping applications to trait-based physiological breeding. Red, Green, Blue (RGB) cameras sense the reflected energy from the plants in the visible part of the electromagnetic spectrum. CCD (charge-coupled device) and CMOS (complementary metal oxide semiconductor) are the most broadly used technologies in image sensors. They can produce a very large number of images in very short periods. Visible light imaging is most commonly acquired due to the low cost of cameras and the wide range of traits that can be derived.

Stereo vision can provide in-depth information from motion techniques by using two mono-RGB cameras. It is an affordable 3D image acquisition system compared with other technologies such as LIDAR (light detection and ranging). Stereo vision performance is affected by changes in the scene illumination and requires stereo–matching algorithms to improve accuracy. Moreover, their performance is adversely affected by the objects' lack of surface texture. Time-of-flight (ToF) camera has been the last imaging device to be incorporated into automatic plant phenotyping. The ToF camera employs near-infrared emitters and measures the distance between the camera's objective and each pixel. ToF is highly suitable for real-time application and allows for precision 3D reconstruction; however, resolution and sunlight are two major factors affecting ToF camera performance.

4.2 Multispectral imaging

Multispectral imaging is widely used for fast, non-destructive measurements of forest tree phenotypic traits. Plants in different growth statuses reflect different spectral signatures. Multispectral cameras capture images from several discrete bands, and the spectral bands may not be continuous (Kolarik et al. 2019). Multispectral cameras commonly include 3–10 spectral bands in the visible and infrared spectral regions. The most used spectral band channels are green, red, red-edge, and NIR (near-infrared). Due to the limited number of available bands multispectral cameras are mainly used for VI (vegetation indices)-based traits.

4.3 Hyperspectral imaging

Hyperspectral imaging typically captures several hundreds of continuous bands within a specific range of wavelengths. The spectral resolution is the main factor that distinguishes multispectral from hyperspectral imagery (Jiang et al. 2021). Compared with multispectral imaging, hyperspectral cameras have a higher spectral resolution, with continuous or discrete spectral bands in the visible and infrared spectral region (Huang et al. 2020). Due to the large volume of data associated with spectral imagery, their operation is more complex. Hyperspectral cameras indirectly assess advanced phenotyping traits, including leaf water content, pigments concentration, and photosynthesis parameters.

4.4 Thermal infrared imaging

Thermal infrared imaging is used to measure radiation in the thermal spectral infrared regions. It may be used to indicate the temperature gradient across the canopy to study plant water stress and stomatal conductance relations. Abiotic or biotic stresses often result in decreased photosynthesis and transpiration rates. Measuring plants temperature by thermal imaging can be a reliable way to detect changes in the physical status of plants in response to different abiotic or biotic stresses (Richardson et al. 2021). In forest tree phenotyping, thermal infrared imaging offers potential application in breeding programs for drought-prone environments by detecting canopy temperature.

4.5 Fluorescence imaging (FLUO)

Fluorescence imaging is commonly used to detect the resilience of a plant's metabolic status. Fluorescence is a phenomenon, when the light is re-emitted by molecules after they absorb radiation in the ultraviolet, visible, and near-infrared spectral wavelengths. Irradiation of chloroplasts with actinic or blue light will produce some remission of absorbed light by chlorophyll. Because using modulated fluorescence requires substantial power for rapid illumination, fluorescence imaging is often used in a controlled environment. The proportion of re-remission light compared with the irradiation depends on the plant's ability to metabolize the harvested light (Martina et al. 2017). The re-emitted light is the fluorescence, and it is a good indicator of the plant's capacity to assimilate actinic light. Furthermore, combining an actinic light source with brief, saturating blue pulses can be used to measure the plant's efficiency of photo-assimilation, non-photochemical quenching, and other physical plant parameters.

Chlorophyll fluorescence is a sensitive indicator of the physiological status of plants and it can be used to detect abiotic and biotic stress in forest trees. The corresponding fluorescence parameters include maximal photosystem II quantum yield F_v/F_m, actual photosystem II quantum yield △F/F_m', chlorophyll index, anthocyanin index. These parameters can be used to carry out studies of plant stress analyses, photosynthetic functions, and chloroplast content estimations. Current fluorescence sensors are focused mainly at the leaf level. Chlorophyll fluorescence at the canopy scale is restricted by sensor and background noise, decreasing the signal-to-noise ratio.

4.6 3D laser imagine and light detection and ranging (LIDAR)

3D laser imaging, including LIDAR, is an active remote sensing technique that uses laser pulse light to directly measure the 3D distribution of plant canopies. It is used for the creation of a cloud of points that reconstructs trees 3D structure. 3D laser imaging creates accurate and detailed 3D models by structured light projection and laser range scanners (Fernando et al. 2017). Satellite-based LIDAR systems are used to measure tree height, canopy volume, and AGB (above-ground biomass) (Chen and Cihlar, 1996; Gwenzi et al. 2017; Kellndorfer et al. 2010). The direct utility of LIDAR application in tree improvement was demonstrated in assessing tree height and crown geometric features in Douglas-fir progeny testing and realized yield trails in British Columbia, Canada (Grubinger et al. 2020; Du Toit et al. 2020, 2021). Recent application using both manned and unmanned flights has allowed the estimation of biomass dynamics of a coniferous forest using Landsat satellite images, together with ground and airborne LIDAR measurements (Badreldin and Sanchez-Azofeifa, 2015; Guo et al. 2018). Aboveground biomass and volume, derived from either upper proportions of a large-footprint full-waveform LIDAR profiles, or statistically modeled from discrete return small-footprint LIDAR point clouds, to be the most commonly extended forest attributes, followed by canopy cover, basal area and stand complexity (Coops et al. 2021). Some 3D laser scanners could fuse color and point cloud, generating colored point clouds. Even for LiDAR which does not have color information, several publications introduced methods to fuse color images and point clouds. Therefore, the major limitation is how to obtain dense point clouds efficiently. For forestry applications, usually, it needs to scan very large areas, so how to obtain 3D point clouds within a short time period is an important factor.

4.7 Nuclear magnetic resonance imaging (MRI)

Magnetic resonance imaging (MRI) is a non-invasive imaging technique that employs radio frequency magnetic fields to construct tomographic images (Boviki, 2005). In plant phenomics, MRI uses nuclear magnetic resonance to generate images and detects nuclear resonance signals originating from stable isotopes (C13 and N15) (Melkus et al. 2011). MRI is also used to visualize 3D structures and metabolites. This method poses a great potential to monitor physiological processes occurring in vivo. In addition, MRI can describe moisture distribution and be applied for the non-invasive quantification of trees or tree organ water content and estimate water diffusion and transport. One of the greatest advantages of MRI is its ability to distinguish different water levels in wood with the help of relaxometry (Zhou et al. 2018).

4.8 Multi-sensor fusion

Multi-sensor fusion refers to the techniques that integrate complementary information from multi-image sensor data. Together with different sensors, they form an integrated system to provide data support and decision-making basis for intelligent forestry management. The new images are more suitable for human visual perception and computer processes such as segmentation, feature extraction, and object recognition. Different imaging sensors are optimized for diverse operating ranges and environmental conditions. However, individual sensors may not receive all the information necessary to detect an object (leaf, plant, and canopy) by human or computer vision.

Multi-sensor fusion is one of the key technologies to improve the perception ability of HTP, and its research is of great significance to HTP application. Compared with the acquisition of single information, based on multi-sensor fusion method, various types of multi-source data are subjected to different operations and processing to extract the characteristic information of the target for analysis and understanding, finally realizing the identification and recognition of the target. An effective combination of such sensors with different features or viewing positions could extend the capabilities of each individual sensor. The integration of multiple sensors will strongly enhance the functionality of sensors in forest tree phenotyping. A multi-sensor observational approach is developed to identify imagery pixels of a black poplar (Populus nigra) breeding population at the canopy level were captured and analyzed by multi-sensor. Implementation of the surface energy balance from data collected with TIR, RGB, and multispectral images afforded phenotyping a whole black poplar progeny exposed to water limitation conditions in the field (Tauro et al. 2022). Most notably, the illustrated work demonstrates a multi-sensor data fusion approach to tackle the global challenge of monitoring landscape-scale ecosystem processes at fine resolution. It can be assumed that increased attention to the integration of multi-sensor with deep-learning methods to improve estimation accuracy for forest tree phenotyping is needed.

4.9 Deep learning in plant image analysis

With the development of high-throughput plant phenotyping techniques, big phenotypic data of various plant optical sensors can collect image data. As images are machine-produced data, but image types and processing procedures may be very different, image analysis plays an important role in genomic and proteomic projects, which include image preprocessing, segmentation, and features extraction (Fernando et al. 2017). There is an urgent need to develop effective approaches to dealing with large-scale image data analysis to explore their biological and physiological mechanisms. Deep learning provides an opportunity to extract useful traits from the complicated phenotypic dataset, bridging the knowledge gap between genotype and phenotype for fundamental research and engineering applications in a breeding program (Hwy et al. 2019). The databases with raw image have to go through deep learning-based data analysis in order to generate results interpretable by humans. For improving the image analysis systems, deep learning has played a key role (Fig. 4). Deep neural networks have many layers which transform input images to outputs (i.e., healthy or stressed) with learning deep features.

Fig. 4

Overflow workflow for processing image-based tree phenotypic data

Recently, a series of publications focused on forest tree phenotyping relating to the following algorithms: Convolutional Neural Network (CNN), Restricted Boltzmann Machines (RBM), Auto Encoder (AE), Sparse Coding (SC), and Recurrent Neural Network (RNN). The published work involved the phenotypic identification and classification over various varieties of trees from tissues, organs, and plant scales singly or combined (Cen et al. 2020). Deep learning showed the most potential capability for morphology and physiological information extraction, image segmentation and identification, and pest detection. Deep learning has been applied successfully in tree species classification (Guan et al. 2015), stock volume estimation (Liu et al. 2019a), tree crown detection (Weinstein et al. 2019; Weinstein et al. 2020), recognition of diseased pine tree (Hu et al. 2020), oak acorn viability recognition (Przyblo et al. 2019), conifer/deciduous classification (Hamraz et al. 2019). Studies also found that the qualitative analysis of abiotic stress could be diagnosed by transfer learning to reduce training time without affecting the model’s prediction capability, especially network architectures with mature applications scenarios manifested stable performance in terms of adaptability and migration based on CNN or integrated with CNN.

Most of the published work is based on the 2D images in tree phenotyping research, such as digital and greyscale images. Such images could be enabled to operate in the deep transfer learning architecture, while such pre-trained transfer networks could not be applied to the 3D datasets, such as hyperspectral images, which are more sensitive to detecting the early-infected plants. In the future, deep neural networks that can be used for 3D images should be the focus (Gao et al. 2020). It can be assumed that researchers will have to pay more attention to integrating multi-sensor data with deep learning methods to improve the features of the image acquisition system.

5 Controlled environment and field HTP platforms with imaging sensors

The rapid advances in phenotyping technologies have recently enabled researchers to collect high-quality and repeatable phenotypic data. HTP platforms can utilize different leaf-level, near canopy, and airborne sensors to acquire multi-source data on a large scale over a short period. Forest trees phenotyping platforms are divided into those used in controlled environments or in the field.

5.1 Controlled environment phenotyping platforms

Some HTP platforms have been applied in environmentally controlled growth chambers or greenhouses, where stable meteorological conditions can be created to obtain high-quality images amenable to further processing. Controlled environment phenotyping platforms have the advantages of high precision, superior repeatability, and negligible interference from external environments. It includes “plant-to-sensor” (tree movement type) and “sensor-to-plant” (sensor movement type) working modes according to the motion state of target plants and sensors during the operation. These environmentally controlled platforms can be used to obtain tree phenotypic information of containerized seedlings or potted saplings.

“Plant-to-sensor” means that the sensor’s position is fixed, and the potted target plants enter the working area through a conveyor belt and other transporting mechanisms. The sensors collect and analyze the data of the target plants in batches. The platform of the “plant-to-sensor” type is relatively well-developed, mainly due to the sample small size and uniform environment; however, the overall efficiency of the plant-to-sensor method is low, and it can affect the plant’s condition (i.e., damage) at the mature stage, causing errors in monitoring.

The "sensor-to-plant" phenotyping platform moves the sensors to the target plant areas to obtain information. This scanning operation mode keeps the position of trees fixed, has less interference to tree growth, and has great flexibility in sensor movement and high measurement efficiency. However, the system poses a great challenge for both hardware integration and software development to collect real-time population data and realize high-throughput phenotypic parameter extraction in the process of sensor movement.

A “plant-to-sensor” platform was applied to assess biophysical traits and drought response in two oak species (Quercus bicolor and Q. prinoides) in a controlled environment. Potted oak seedlings placed on an automatic conveyor belt entered sequentially different imaging chambers with RGB and hyperspectral cameras. Robotics in the system transfer the plants around and facilitate imaging by lifting and rotating the plants. By quantifying oak seedlings’ growth and development [plant height, projected leaf area (LA), plant/canopy width, ConvexHull, and plant aspect ratio], the species’ response to drought was evaluated (Del-Campo-Sanchez et al. 2020). Additionally, a “sensor-to-plant” phenotyping platform was built in the growth chamber to evaluate the growth rate of containerized tree seedlings during the pre-cultivation phase following seed germination. Seeds of four tree species (Fagus sylvatica., Quercus ilex, Picea abies, and Pinus sylvestris) were analyzed by collecting stereoscopic RGB images at regular time intervals. Comparative analysis of these images enabled calculating the increments of seedlings' height and leaves greenness percentage (Moran-Duran et al. 2020).

The greenhouse-based analyses to quantify certain traits of forest trees face several major challenges. One reason is that many important traits occur only when plants are grown in the field. For example, the steepness of a slope affects tree growth through the differential incidence of solar radiation, wind velocity, and soil type, which is known as the slope effect. A steep slope is also susceptible to rapid surface runoff and soil erosion. The influence of this abiotic factor on trees growth and distribution is significant in the field (Måren et al. 2015). However, it is difficult to mimic the slope effect in the greenhouse as all plants are grown on a flat surface. Seedlings in greenhouses are grown in artificially controlled environments that might significantly alter the normal pattern of trees growth and development.

5.2 Field phenotyping platforms

Field phenotyping includes development at two contrasting levels of resolution, tree-based, and area-based, and both are primarily focused on characterizing tree growth. Tree-based phenotyping aims to quantify and isolate the effects of competition and environment on individual trees' growth and then accurately identify what trees with particular genotypes are within forest stand level. Genetic testing can then be applied to those trees, and specific parents are identified to improve the gene pool used in breeding. Area-based phenotyping aims to characterize stand-level performance, identifying superior combinations of seed sources, stands, and silvicultural treatments. It will also allow managers to maximize production by identifying the optimal combination of seedlot, site, and silviculture within their forest.

The difficulty with outdoor platforms increases due to limitations in the actual image acquisition devices and the uncontrolled conditions that directly affect image quality, especially for forest trees, due to the intrinsic difficulties in measuring long-lived tall trees in their natural environment. Regardless of recent developments, the application of HTPP in forestry is still in its infancy.

Recently, field phenotyping platforms have been established to assess plants’ traits in fixed-site experiments at a sufficient throughput. Thus, recent research projects have involved setting mobile field phenotyping platforms often termed “phenomobiles” (Deery et al. 2014). In addition to the field soil conditions limiting the operations, many phenomobiles require manual operation (Zhang et al. 2020) as they cannot be driven through the experimental fields at high speed, constraining the throughput and impairing efficiency (David et al. 2016).

Due to the limitation of acquisition speed and area, the vehicle-mounted system has a certain time asynchrony for large-scale phenomic data acquisition for forest trees, which will lead to errors of canopy parameters. Aerial phenotyping platforms are increasingly considered as an alternative option to overcome the limitations associated with ground-based phenotyping platforms. Aerial phenotyping platforms may be classified as MAVs (manned aerial vehicles), UAVs (unmanned aerial vehicles), and satellites. Aerial phenotyping platforms enable the rapid characterization of wide forestry areas within hours.

In the past 20 years, UAV platforms have been widely used in forest resource surveys and disaster assessment (Mukherjee et al. 2019). Compared with the traditional airborne platforms, the small payload and short measurement duration of UAV platforms are the major limitations of their application. UAVs are commonly equipped with customizable sensor payloads for phenomic data collection. Miniaturized light-weighted airborne sensors have been developed to meet the limited payload capability of small UAV platforms. Multiple types of cameras and other sensors are now available. These platforms are suitable for measuring spectral and morphological information, including plant height and canopy surface profiling. UAV phenotyping platforms extract morphological information by LiDAR and SFM (Structure-from-Motion). The basic idea of SFM is to make the camera move to acquire several images from multiple perspectives as a UAV flies over a field. Through mathematical analysis of the object in the image sequence, the 3D growth parameters of trees are calculated, and the 3D plant traits, including population, tree height, DBH, flowering time, and forest growing stock, are subsequently obtained.

All phenotyping platforms have distinct advantages and limitations for field-based forestry applications. Phenomobiles have a limited coverage area, so it directly affects the number of samples. The payload capacity of the drone limits UAV's application in phenotyping. Because the electric motor must directly generate all of the required lift to keep the vehicle in the air, UAV platforms tend to have a small payload (sensor) capacity. They are problematic to be operated in windy conditions, and they produce “downwash”—air turbulence below them that can strongly affect canopy structure. They also fly at low speed and exhibit a short flight time (between battery charges). UAV platforms typically are far from the canopy to be investigated. Therefore, the resolution of images is often low compared to the proximal platform but still significantly higher than airborne or spaceborne platforms.

It is clear that high-throughput field phenotyping faces several challenges. Firstly, high-quality plant phenotypic data is difficult to obtain in natural field conditions. For example, the wind will blur plant images and unsuitable for quantitative analysis. Rapidly fluctuating radiation levels (due to clouds) could significantly reduce the accuracy of the passive-type spectroscopic measurements (Ge et al. 2016). Secondly, a key challenge of using UAVs for deriving tree structure/DBH, etc., in forests is the exact localization of trees from the ground to link them to UAV signals. Thirdly, the platform should measure and integrate many phenotyping traits that take different data formats (point measurement, spectra, and images).

6 Application of HTP platforms

Automated HTP platforms equipped with imaging sensors can enable the evaluation of larger populations, which increases selection intensity and improves selection accuracy. Currently, phenotyping might occur in large forests on the landscape level and individual trees in growth chambers. Recent research that has been carried out covers trees phenotyping at a large scale, from various genotypes using simple and repeatable technology to individual trees using precise and accurate methods. However, this remote sensing approach on single trees has scarcely been applied.

6.1 Survey of actual inter and intraspecific variability

A better understanding of species coexistence and community dynamics may benefit from more insights on trait variability at the individual and species levels. Investigating the phenotypic characteristics of trees offers an excellent system to understand the relative impact of intraspecific and interspecific variability on community assembly, due to trees’ phenotypic plasticity, and the strong influence of environmental variables have on their spatial distribution and individual performance. The main objective of the survey is to measure the growth status (height, DBH, crown density, leaf nitrogen content, and chlorophyll content), wood volume, biomass, or species diversity. Most forest inventories are based on analysis of sampling plots, and the results are used to infer the global parameters of the forest cover under study. HTP platforms have great potential to survey forest resources over large spatial scales.

The application of aerial LiDAR systems (aerial laser scanners, ALS) to forest measurements is operative for evaluating forest vegetation cover and its characteristics. ALS can simultaneously provide horizontal and vertical information on canopy structure. Specifically, the combined use of ALS and plot-level information has effectively increased in the last few years and has proven effective for extracting forest inventory parameters (Vastaranta et al. 2013; Wulder et al. 2013). However, ALS data is not yet extensively used for strategic forest inventory assessments at a national level. Instead, they are used operationally for stand-level forest management inventories (Barret et al. 2016). Several studies provide different approaches to the individual tree or plot-level phenotyping traits of the vegetation cover from terrestrial HTP platforms with laser scanning (Cabo et al. 2018; Yan et al. 2018).

The recent development of high-resolution spectral sensors carried by airborne and space-borne devices makes foliage spectral traits viable for mass phenotyping in forest trees. HTP platforms such as NASA's airborne visible/infrared imaging spectrometer program (AVIRIS) provide high spectral resolution (224 bands) across a large spectral range. These advanced platforms enable the accurate measurement of a suite of canopy foliar traits that play key roles in a survey of actual inter and intraspecific variability. Combining imaging spectroscopy (AVIRIS) data with genetic, biochemical, microbial, and biogeochemical data helps determine how genetic variation influences below-ground processes at the landscape scales (Madritch et al. 2014). The survey of actual inter and intraspecific variability provides scientific data for the species diversity maintenance mechanism, population competition research, and proper management.

6.2 Test for the adaptation of genotypes and species to abiotic stress

High throughput phenotyping should be used to test trees’ response to future climate change scenarios and for breeding stress-resistant varieties. Forest trees are sessile and continue to develop over many growing seasons. Field-grown trees are routinely exposed to environmental stress. Abiotic stresses, such as drought, waterlogging, salinity, wind, heat, and cold, adversely affect tree growth and thus forest productivity and play a major role in determining the geographic distribution of tree species. The development of stress-resistant cultivars is one of the challenging tasks for tree breeders due to its complex inheritance and polygenic regulation. Evaluating genetic material for abiotic stress tolerance is complicated due to its spatiotemporal interactions with environmental factors. The conventional breeding approaches are costly, lengthy, and inefficient in achieving the expected gain in abiotic stress tolerance. Tree response and tolerance to abiotic stress are complex biological processes that are best analyzed at a system level using genotype and phenotype (Harfouche et al. 2014). The use of HTP platforms extends the suite of traits that can be measured and provides a better understanding of stress tolerance.

HTP platforms help us understand how forest trees adapt to harsh environmental conditions. The HTP platform was applied to investigate the drought response of two coexisting deciduous tree species (Quercus cerris and Fraxinus ornus) in a natural mixed forest, and the results have shown that chlorophyll fluorescence measurement is particularly suitable for phenotyping the drought stress response of adult trees in the field (Salvatori et al. 2016). An aerial imaging platform with a thermal camera detected palm trees and pure-canopy pixels. An automatic procedure was suggested to detect palm canopy from aerial thermal images, and a semi-automatic method was proposed for further detection under water stress (Gauthier & Jacobs, 2019).

The success of genomic assisted breeding (GAB) depends upon the precision in marker-trait association and estimation of genomic estimated breeding values (GEBVs), which mostly depends on coverage and accuracy of genotyping and phenotyping (Arenas et al. 2021). A wide gap between the discovery and practical use of quantitative trait loci (QTL) for improvement has been observed for many important traits. Such a limitation could be due to the low accuracy in QTL detection, mainly resulting from low marker density and manually collected phenotypes of complex traits (Bhata et al. 2020). Accurate and precise phenotyping using HTP platform can improve the precision and power of QTL detection. Therefore, HTP can enhance the practical utility of GAB along with a faster characterization of germplasm and breeding material (Fig. 5).

Fig. 5

Schematic diagram shows the importance of phenotyping and genotyping as key players in gene discovery and stress-resistant breeding

The benefits of a successful seedling establishment and growth can extend beyond the current growing season. Therefore, early plant selection for desirable morphological and physiological traits and the ability to quantify the changing competitive relationship between genotypes and species under global climate change are critical. Roots, stems, and leaves were observed using a platform equipped with laser scanning to research poplar adaptation to drought stress (Zhou et al. 2020). HTP platform with RGB and hyperspectral cameras were successfully used to evaluate seedlings of two 1-year-old oak species, Q. bicolor and Q. prinoides, where the former showed faster initial growth earlier in the season than the latter under drought stress, resulting in a larger leaf area and seedling dimension. This information confirms the inherent differences in seedling growth and shade tolerance between the two species, affecting species selection for sustainable forest management (Mazis et al. 2020).

Due to the ongoing global climate change, abiotic stresses pose a serious threat to forest productivity worldwide, affecting tree growth and survival. HTP platform enables the collection of phenotypic traits. It can promote the successful selection of stress-resistant genotypes and species effectively, which is important for early plant selection for forest management and tree improvement programs. To make progress at the rate required by global demand in a changing climate, traditional phenotyping measurements must evolve to accurately, quickly, and reliably obtain more scalable measurements with high resolution. Breeders today have HTP platforms as useful tools for selecting trees with new traits to face increasingly challenging climate. Because forest trees are sessile and continue to develop over many growing seasons, mechanisms have evolved that allow trees to respond to changes in environmental conditions (Ludovisi et al. 2017). It is now recognized that these technologies need improvements to face new challenges, such as predicting complex plant phenotypes (Varshney et al. 2021) and assessing the impact of climate change on tree production, and developing new types and management practices based on underlying genotype-environment-management interactions. To face these challenges, breeders will have to adapt, and a key element in this will be the development of “future-proof” trees. These new selections will withstand future climate conditions and will have to make very efficient use of scarce resources such as water, and nutrients (Hein et al. 2021).

6.3 Pest and disease detection

Plant pest and disease detections quantify the visible signs or symptoms of stress and its progression on an individual tree at the leaf, canopy, plot, and stand levels. Visible symptoms in the foliage of broadleaves and conifers play an important role in detecting and scaling up the effects. Initial attacks are not visible to the human eye (Ortiz et al. 2013). Deployment of HTP platforms and standardization of visual assessments have improved the accuracy and reliability of stress assessment compared to the traditional labor-intensive manual measurement.

Automated, high-throughput digital imaging techniques by UAV can be used in phenotypic analysis of forest diseases and pests to collect data at multiple time points. Images generated can derive quantitative phenotypic data and improve the reproducibility of information analysis. The high-resolution hyperspectral imaging based on UAV has been of high practical value for forest health management, for instance, indicating a pest outbreak in time (Coops et al. 2010; Nurminen et al. 2015; Kantola et al. 2010). A processing approach for analyzing spectral characteristics for high spatial resolution hyperspectral image data in a forested environment was developed to identify damaged trees. The point clouds measured, using dense image matching, enabled the classification of trees into healthy, infested, and dead categories (Näsi et al. 2015).

Forest tree stress phenotyping is essential for selecting biotic stress-resistant varieties and developing better biotic stress-management strategies. In the future, ground images will be combined with aerial photos to monitor forest health status. HTP platform can play an essential role in integrative resistance breeding through the non-destructive screening of large numbers of plants for plastic phenotypic responses to biotic stresses.

6.4 Phenological measurement

To evaluate the interaction between vegetation and atmospheric processes under rapid climatic change, it is necessary to accurately detect spatial and temporal variations in forest tree phenology, such as the time of flowering, leaf coloring, leaf expansion, and leaf fall. Recent reports suggest the importance of accurately detecting tree phenology’s spatial and temporal variability under rapid climate change (Mark & Liang, 2013; Richardson et al. 2013). The main objective for breeders has been to develop new genotypes with improved desirable traits related to tree architecture and high yield, which is directly related to reproduction and flowering. Variability in flowering among individuals directly impacts their fitness, but how reproductive phenology is affected by the size of the individuals needs further research (Mauricio et al. 2013). Long-term continuous phenological observations and analyses of both individual tree species and the whole canopy depend on the HTP platforms.

Although aerial phenotyping platforms with satellite images permit relatively inexpensive, high-frequency phenological monitoring over wide areas, images' low resolution prevents achieving the goal of monitoring individual trees or species. In addition, the lack of daily-resolution data prevents capturing important short-term changes such as reproductive and vegetative flushing. Long-term continuous phenological observations can solve these problems. An HTP platform with an RGB camera was installed on a crane's side to capture daily images of the forest canopy to observe phenology and the result has indicated that the temporal patterns of red, green, and blue channels extracted from phenological images can detect trees characteristic (Nagai et al. 2016). HTP platform can also reveal the year-to-year variability in the timing of flowering, leaf expansion, and leaf-fall of the whole canopy and individual trees (Watson et al. 2019; Inoue et al. 2014). An efficient RGB-UAV-based platform was used to detect the flowering density and blooming periods. This HTP platform could be adapted to provide critical support to promote commercially feasible applications of phenological measurement in forestry phenotyping for researchers or breeding and seed orchards. Results showed an individual tree's flowering and canopy evolution on different dates (Francisca et al. 2019). The HTP platform was useful for detecting the variability in phenological stages of tree varieties by mapping and quantifying the height, volume, flowering dynamics, and flower density of every tree.

Recent studies showed the utility of analyzing temporal patterns based on data extracted from images to evaluate complicated temporal variations of tree phenology accurately (Geng 2021). These images could eventually be analyzed together with the meteorological data, providing an increasingly useful tool for assessing the sensitivity of tree phenology to climatic change.

In summary, HTP platforms should be adopted by tree breeders as a powerful phenotyping tool. The rapid development of miniaturized and mobile technologies has provided economic and powerful sensors for forest tree phenotyping with high-resolution images. Therefore, affordable and efficient HTP platforms will become the common choice for tree breeding and seed orchard programs. As sensors have become lighter and smaller, they can be integrated with different ground and aerial phenotyping platforms, facilitating effective and efficient phenotyping.

7 Conclusion and perspectives

Conventional breeding has successfully improved various traits that impact tree growth, such as crown architecture and partial abiotic or biotic stress resistance. Because of their long generation intervals, large genomes, and the lack of well-characterized mutations for reverse-genetic approaches, the continued improvement of forest trees is slow. Nevertheless, breeding possibilities have been broadened by forest tree phenotyping.

For tree breeding and forestry management, phenotypic analysis is the key to understanding gene function and environmental effects. Phenotyping a tree’s traits could identify better-growing plants, so breeders select superior genotypes and analyze the impact of environment and silviculture. Geneticists will need to work more closely with physiologists, ecologists, and engineers to develop informative, precise, and standardized HTP technologies.

Phenotyping facilities actively contribute to the generation of high-dimensional, richly informative datasets on trees. Here, we reviewed recent advances in sensors and HTP platforms applied in forestry, emphasizing the challenge connected with phenotyping systems and the application of HTP platforms. With the support of phenotyping platforms, we can guide germplasm selection at the early stage of breeding, evaluate field performance, and detect the occurrence of stresses. Prediction (before selection) of phenotype at the older stage with image analysis at the juvenile stage is needed in advanced breeding programs. Therefore, the high-throughput phenotypic assay can accelerate the breeding process and provide important support data for resource regulation and management decision-making in precision forestry. Besides the acceleration, these images could give access to new phenotypes that were not accessible until now at the breeding level, i.e., for a large number of trees (ecophysiological traits for example).

To build an efficient integrated forest tree phenotyping community, multidisciplinary collaborations between forest geneticists and engineers /technology providers are needed. Ongoing efforts are required in the development and application of high throughput phenotyping technology to combine with genome-wide selection (WS), quantitative trait loci (QTL), and genome-wide association study (GWAS) to identify the function of causal genes. HTP will also be proposed as a tool to indirectly capture endophenotypic variants and compute relationship matrices for predicting complex traits and give rise to this new approach of "phenomic selection" (Rincent et al. 2018). As new and improved sensors are developed, HTP platforms with high-resolution, high-accuracy, and affordable prices will accelerate tree breeding. In the future, phenotyping these complex traits will require sensor advancement, high-quality imagery combined with deep learning methods, and efforts in transdisciplinary science to foster integration across disciplines.