Deep Learning in Plant Phenological Research: A Systematic Literature Review
Negin Katal conducted a systematic literature review to analyze all primary studies on deep learning approaches in plant phenology research. The paper presents major findings from selected 24 peer-reviewed studies published in the last five years (2016–2021).
Research on plant phenology has become increasingly important because seasonal and interannual climatic variations strongly influence the timing of periodic events in plants. One of the most significant challenges is developing tools to analyze enormous amounts of data efficiently. Deep Neural Networks leverages image processing to understand patterns or periodic events in vegetation that support scientific research.
“[…]deep learning is primarily intended to simplify the very time-consuming and cost-intensive direct phenological measurements so far.”
Technological breakthroughs powered deep learning approaches for plant phenology within the past five years. Our recently published paper describes the applied methods categorized according to the studied phenological stages, vegetation type, spatial scale, data acquisition. It also identifies and discusses research trends and highlights promising future directions. It is freely available here: https://www.frontiersin.org/articles/10.3389/fpls.2022.805738/full
To understand how scientific research on phenology is done, the different methods to retrieve phenological data need to be clear:
- Individual based observations are for example human observations of plants, cameras installed below the canopy or even reviewing pressed, preserved plant specimens collected in herbaria over centuries and around the globe.
- Near-surface measurements cover research plots on regional, continental, and global scales and are done for example via PhenoCams, near-surface digital cameras located at positions just above the canopy, or drones.
- Satellite remote sensing is done via satelite indices, such as spectral vegetation indices (VIs), enhanced vegetation index (EVI), and more.
Overview of methods monitoring phenology.
Key findings:
The reviewed studies were conducted in eleven different countries (three in Europe; nine in North America, two in South America, six in Asian countries, and four in Australia and New Zealand) and across different vegetation types, i.e., grassland, forest, shrubland, and agricultural land. The vast majority of the primary studies examine phenological stages on single individuals. Ten studies explored phenology on a regional level. No single study operates on a global level. Therefore, deep learning is primarily intended to simplify the very time-consuming and cost-intensive direct phenological measurements so far.
In general, the main phenological stages are the breaking of leaf buds or initial growth, expansion of leaves or SOS, the appearance of flowers, appearance of fruits, senescence (coloring), leaf abscission, or EOS. More than half of the studies focused either on the expansion of leaves (SOS) or on the flowering time.
Across the primary studies, different methods were used to acquire training material for deep learning approaches. Twelve studies used images from digital repeat photography and analyzed those with deep learning methods. The publication shares in-depth information of the different types of digital photography suitable to provide those training data.
Furthermore, it categorizes, compares, and discusses deep learning methods applied in phenological monitoring. Classification and segmentation methods were found to be most frequently applied to all types of studies, and proved to be very beneficial, mostly because they can eliminate (or support) tedious and error-prone manual tasks.
Future trends in phenology research with the use of deep learning
Machine learning methods need huge amounts of data to be trained. Therefore, increasing the absolute number of collected data is one of the key challenges – especially in regions or countries that lack traditional phenological observing networks so far. The paper describes methods and tools that will become important levers to support this kind of research, for example:
- Installing cameras below the canopy that automatically take pictures and submit them over long periods of time are one way to tackle that.
- PhenoCams prove to be a new and resourceful way to fuel further research: With indirect methods that track changes in images by deriving handcrafted features such as green or red chromatic coordinates from PhenoCam images and then applying algorithms to derive the timing of phenological events, such as SOS and EOS. We expect many more studies to appear in the future evaluating PhenoCam images beyond the vegetation color indices calculated so far.
- Citizen Science data from plant identification apps such as Flora Incognita prove to be a long-term source of vegetational data. These images have a timestamp and location information and can thus provide important information about, e.g., flowering periods, similar to herbarium material.
We see that the study of phenology can easily and successfully exploit deep learning methods to speed up traditional gathering and evaluation of information. We, as a research team, are very proud to be a part of that and invite you to play a vital role – in using Flora Incognita to observe the diversity and change of biodiversity around you.
If you have any questions to us regarding our research, don’t hesitate to reach out! You can find Negin Katal on Researchgate and Twitter (), for example.
Publication:
Katal, N., Rzanny, M., Mäder, P., & Wäldchen, J. (2022). Deep learning in plant phenological research: A systematic literature review. Frontiers in Plant Science, 13. https://doi.org/10.3389/fpls.2022.805738
