Plant identification app “Flora Incognita” honored with Thuringian Research Prize

/in , /by

One third of the plant species in Germany is listed as endangered, tendency increasing. At the same time, the number of people with species knowledge is continuously decreasing. But how can we protect species that we don’t recognize? The Flora Incognita research project combines smartphones, artificial intelligence and citizen participation in an app that interactively and automatically identifies plants based on image recordings. With every successful application, the app learns and improves its recognition accuracy. At the same time, the records of the identified species and locations create valuable data sets to answer questions of species protection and biodiversity. More than 1 million people, from enthusiastic laypersons to biology professors, are already using the free app. The interdisciplinary project team from the Max Planck Institute for Biogeochemistry in Jena and the Technical University of Ilmenau was honored for its development with the Thuringian Research Prize in the category of applied research.

Together with climate change, the loss of biodiversity is one of the greatest threats to humanity. Plants form the basis of the food webs on our planet. If their composition changes, the entire ecosystem with its important functions for animals and humans is affected. A central requirement of species protection is therefore the comprehensive survey and monitoring of plant biodiversity. However, fewer and fewer people, even academically trained biologists, are now able to reliably identify a larger number of plant species and place their occurrence in an ecological context. How can one promote the protection of biodiversity if it is practically unknown to the population?

“For this reason, we developed a method for interactive and automatic plant identification with a smartphone app,” explains biologist Dr. Jana Wäldchen, project manager at the Max Planck Institute for Biogeochemistry in Jena. “After all, what could be more obvious than to combine our botanical species knowledge with the latest machine learning technologies and the constantly growing availability of mobile devices such as smartphones and tablets? With the smartphone’s camera, users take a picture of the flower and, if necessary, the leaf of the unknown plant, and within seconds the app provides them with information about the recognized species – not only the common and botanical names, but also further information.

Scientists have been working for a long time on the automatic recognition of plants using images. The models developed so far have been able to extract botanical traits such as leaf and flower shapes or flower colour, but the trait selection itself was often problematic. For a reliable determination, users had to select the crucial traits individually, which proved to be very labor-intensive and required expert knowledge. Classifications of more than 100 species were hardly possible.

Also, there was no procedure so far that could independently extract plant characteristics from an image without a predetermined pattern recognition programmed by humans. “In the last five years, we have achieved a fundamental breakthrough in this area through deep learning artificial neural networks and in the field of image processing,” confirms computer scientist Prof. Patrick Mäder (JP), project manager at the TU Ilmenau. The image recognition method of the Flora Incognita App is based on an artificial neural network. Fed with more than two million plant images, the Flora Incognita network learned the different plant characteristics of over 4800 species. To identify plants as accurately as possible, Flora Incognita determines the location and automatically compares it with stored data sets on plant occurrence, soil and climate, which further improves the hit rate. By programming a situation-dependent identification process, the app can also request further specific information or images, depending on the first hit rate. On top of this, there had been extensive scientific studies on which different image perspectives are most important for successful plant determination.

The free-of-charge and ad-free Flora Incognita App was already released in spring 2018. Since then, the app has been downloaded by more than 1 million users and performed over 8 million searches. Flora Incognita is not only used by laypersons, but also by botanical experts, who rate the app extremely positively and recommend it.

Science also benefits from the app: recording identified species and their locations creates extremely valuable data sets with which questions of species protection and biodiversity can be investigated. In the long run, the data from the Flora Incognita app will enable new insights into questions such as: When and where do which species flower? How much do the traits vary within a given plant species? How do the composition and location of plants change in the context of climate change and land use? First results on such topics have already been presented at scientific conferences.

The Flora-Incognita project is being driven forward by an interdisciplinary team of scientists from biology, physics, media technology and computer science at the Max Planck Institute for Biogeochemistry, Jena, and the Technical University of Ilmenau. It has won several awards and the team successfully secured third-party funding. The project was supported by the Federal Ministry of Education and Research, the Federal Agency for Nature Conservation with funds from the Federal Ministry for the Environment, Nature Conservation and Nuclear Safety and the Foundation for Nature Conservation Thuringia from 2014 to 2020. Since autumn 2019, the team has been working on the follow-up project Flora Incognita++, funded by the Federal Agency for Nature Conservation with funds from the Federal Ministry for the Environment, Nature Conservation and Nuclear Safety, and by the Thuringian Ministry for the Environment, Energy and Nature Conservation.

The Thuringian Research Prize is awarded annually by the Thuringian Ministry of Economics, Science and Digital Society (TMWWDG) in two categories: basic research and applied research. The award ceremony and congratulations by Minister Wolfgang Tiefensee took place virtually this year.