Project Overview

Species are the basic unit of biological systems.

The ability to determine species correctly and quickly is the basis for a variety of scientific topics. Without species knowledge, taxonomic and ecological questions cannot be answered. In addition, for al kind of biodiversity studies, the occurrence and frequency of species and species assemblages are the basis. Ultimately, practical recommendations for nature conservation are also based on species knowledge. Even for experts, it is not always easy to determine plant and animal species using conventional identification books. This time-consuming and complex task is considerably error prone, depending on the condition of the object and the experience of the person. People dealing with the issues described above therefore need more efficient methods to determine species, especially in the context of the continuing loss of biodiversity. In addition, there are fewer and less educated people with the corresponding specialist knowledge, and the these are sometimes spread all over the world.

New solutions are needed.

Thus, in the age of digitization, computer-supported systems may offer a reliable alternative to the traditional approach to taxonomic identification. Such systems can learn characteristic features of species and to a certain extent automatically recognize them. Due to their independence and permanent availability, it is ultimately also possible for laymen to make use out of specialist knowledge.

As part of the „Flora Incognita“ research project, a method for semi-automatic, interactive plant identification using mobile devices is to be developed, combined with automatic mapping of the respective objects.

Our approach

Modern artificial intelligence techniques are used to perform this complex task.¬† An image set of an individual is created with the camera of the smartphone. Here, the plant itself and additional plant parts relevant for the determination are taken up from clearly defined perspectives. The resulting data set allows an image-based taxon determination based on so-called „Deep Learning“ algorithms possible. These Convolutional Neural Networks (CNNs) are a machine learning method and are widely used in image recognition. They are modeled after the human brain and consist of a large number of cascaded levels with filters and several million neurons (more information on page „How computers learn plants). To train the networks, a large image database of all wild growing tree, shrub, fern and flowering plants in Germany is being built up. It currently contains more than 700,000 images for 2770 taxa, and the number is rising. Interested citizens, so-called Citizen Scientists, support the research team by using the „Flora Capture“ app to take standardized plant images and transmit them to a server at the TU Ilmenau or make their private image collections available. In order to verify and narrow down the results of the image-based determination, site queries of the user are recorded, but also occurrence data of plant species from central biodiversity databases are included and correlated in a prediction model. In addition, the user is asked – if necessary – questions about characteristic features of the plant species, which are answered in an interactive process using graphic icons. By combining automatic image recognition and observed characters, contributed by the users, a plant is in most cases determined down to species level. Taxa identified in this way are transmitted together with their location, time of observation and additional data to central databases of nature conservation authorities and research institutions, where they can then be made available to interested parties and authorities in an open platform.

The Flora-APPS

The project goal of developing a semi-automated device for determining plants is divided into two different components. A separate application was developed for each of them, which will eventually be combined in the Flora Incognita application. First of all, the Flora Key App represents an innovative implementation of classic identification keys. Graphic icons can be used to select plant characteristics and thus guide the user to the determination process. Possible errors are taken into account and dead ends in the determination are avoided. The Flora Capture App, on the other hand, serves primarily to digitally capture plant images sets. An individual is photographed from defined perspectives and combined into a data set with metadata on location, date and phenology. Finally, Flora Incognita combines the automatic determination of plant images and determines questions of characteristics that still have to be answered manually in order to be able to separate optically similar species from each other. With this synergy we try to approach the „botanical view“: artificial intelligence recognizes the rough appearance, while specific characteristic questions allow the exact determination of the species. In addition, environmental data are used to assess whether the taxa in question can potentially occur at the observed site at all.

Flora Key App

Flora Capture App

Flora Incognita App

Who makes this possible?

This interdisciplinary project has been funded since August 2014 as a joint project between the TU Ilmenau and the Max Planck Institute for Biogeochemistry in Jena by the BMBF, BfN and the Thuringian Nature Conservation Foundation. A team of scientists from the fields of biology, physics, media technology and computer science are working together to master the challenges of developing such an application. The cooperation between university and non-university research, as well as the groundbreaking combination of ecology, geosciences and artificial intelligence becomes clear here with exemplary character. The project was awarded as an official project of the „UN Decade of Biological Diversity“. The award is given to projects that work in an exemplary manner to conserve the world’s biological diversity. The semi-automatic recognition of wild flowering plants in Thuringia using a smartphone is intended to raise awareness of biodiversity among the population and ultimately contribute to understanding, preserving and protecting it.

Literature

Why is species knowledge important?

How computer learn to understand plants?

How can I contribute?