Flowers, leaves or both? How to obtain suitable images for automated plant identification

Flowers, leaves or both? How to obtain suitable images for automated plant identification
Flowers, leaves or both? How to obtain suitable images for automated plant identification

Link to the article: https://rdcu.be/bLwSl

We are pleased about the publication of our new paper of the same name.

The aim of this study was to find out from which perspectives plants should be photographed in order to provide a most accurate automated determination. Additionally, we wanted to know how many observations per species are necessary to achieve a sufficiently accurate result.

We selected 100 different plant species common in Germany. For each of these species, 100 images were to be taken from 5 different perspectives. This happened from March to August 2018. Thanks to the many untiring users who supported us here:

At the end it became a bit dramatic behind the scenes. Some species bloom only very for a short period, so searching and finding became a race against drought. But sometimes even such simple things as a lawn mower got in our way. Unfortunately, we had to discard some of the selected species for the study:

   

Ultimately, over 100 species with 100 observations each have come together, yielding a total of more than 50000 images, which were collected for this study. 

Our most important tool was the Flora Capture App. Each capture observation consists of at least five images (= perspectives). We wanted to know: Is all this effort actually necessary? Our results clearly show: Yes! 

  • Taken all perspectives together, we achieve a recognition rate of an impressive 97%.
  • With the best single perspective (flower lateral) we already reach 88%.
  • With three images (that’s why Flora Incognita needs exactly these) you can achieve almost 96% – as we consiter this to be a good compromise between effort and accuracy.

It should be mentioned that these results refer to a data set of 101 species. We think, however, that the results are also largely transferable to more species – if the images are correspondingly good.This statement relates to both the training images and the image you take for the identification.

 For a good result you can tweak two things. Having the determination process in mind: Could you identify plants from a distance and blurred using a determination key? Certainly not always. Digital plant identification requires the same dedication and attention to the detail.

  • Get as close to the plant as possible. Focus on the selected part of the plant. Here you can see an example of how clover species can (not) be detected.
  • Create good capture observations (see illustration). Unfortunately, our total image data set of over 4800 species is not as beautyfully balanced as the one used in the study. Especially for rare, small and less conspicuous species we do have only few or even no structured capture observations. Become a part of the Citizen Science community and help us collecting good images for us to improve the accuracy of the Flora Incognita App!

PS: Take advantage of your smartphone’s zoom function – get as close as possible as long as the picture remains sharp. Can you see small details in the photo? Very good. The more pixels belong to the plant you want to identify, the better.

This helps us collecting tiny flowers or leaves:

Hold a finger or your hand in the same plane where the part of the plant you want to photograph is situated. Focus your hand, it’s usually easy. Then take your hand away and you will see that the lens is still well aligned to focus on the small object.

With a little practice, you’ll even be able to take good photos of grasses. We’re already working on help and improvements for close-ups. Until then, the described procedure often does trick.