Date Published: February 24, 2017
Publisher: Public Library of Science
Author(s): Marco Seeland, Michael Rzanny, Nedal Alaqraa, Jana Wäldchen, Patrick Mäder, Yudong Zhang.
Steady improvements of image description methods induced a growing interest in image-based plant species classification, a task vital to the study of biodiversity and ecological sensitivity. Various techniques have been proposed for general object classification over the past years and several of them have already been studied for plant species classification. However, results of these studies are selective in the evaluated steps of a classification pipeline, in the utilized datasets for evaluation, and in the compared baseline methods. No study is available that evaluates the main competing methods for building an image representation on the same datasets allowing for generalized findings regarding flower-based plant species classification. The aim of this paper is to comparatively evaluate methods, method combinations, and their parameters towards classification accuracy. The investigated methods span from detection, extraction, fusion, pooling, to encoding of local features for quantifying shape and color information of flower images. We selected the flower image datasets Oxford Flower 17 and Oxford Flower 102 as well as our own Jena Flower 30 dataset for our experiments. Findings show large differences among the various studied techniques and that their wisely chosen orchestration allows for high accuracies in species classification. We further found that true local feature detectors in combination with advanced encoding methods yield higher classification results at lower computational costs compared to commonly used dense sampling and spatial pooling methods. Color was found to be an indispensable feature for high classification results, especially while preserving spatial correspondence to gray-level features. In result, our study provides a comprehensive overview of competing techniques and the implications of their main parameters for flower-based plant species classification.
Although flowering plants play a key role in terrestrial ecosystems, humans increasingly lack the ability for their classification . In addition, the classical way of plant classification, i.e., following a single access identification tree of dichotomous keys, is a complicated and tedious procedure for non-experts. However, due to tremendous achievements in the fields of computer vision and machine learning, automated image based classification promises an easy and fast way for plant classification. Using leaf images for this task was extensively investigated in previous studies, e.g., [2, 3]. Whereas leaves can be found at almost any time throughout a year, the acquisition of suitable leaf images poses difficulties as foreground segmentation is required for extracting discriminative shape parameters. Furthermore, such shape parameters are in most cases only valid for a certain leaf type, i.e., plain single leaves.
For training a classifier such as a Support Vector Machine (SVM), it is required to quantify the information contained in every image into vectors of fixed length, i.e., image representations. The dimensions of the image representation span the descriptor space in which the classification process is reduced to a similarity measure between descriptors. However, given a set of images the amount of local features per image can be very different as it depends on the amount of structurally prominent regions as well as the image size. Thus, a processing pipeline is utilized that aggregates the visual information of many local features extracted from an image into an image representation (see Fig 1).
Whereas in the early days of computer aided plant classification researchers often reported results on unpublished datasets, the publication of the Oxford Flower 17  and Oxford Flower 102  datasets along with splits and accuracy score definitions today allows for comparing results of different studies utilizing different methods. These datasets are an accepted benchmark for fine-grained flower-based classification tasks. We reviewed relevant publications on flower-based plant classification using local features and summarized their methods for each step of the processing pipeline (see Fig 1) sorted by year of publication in Table 1. The table solely lists the methods, but they are briefly described in the next section.
Given the overview on previously studied methods for each processing step (see Fig 1) in section Related Work, we conclude that various general purpose features and methods were successfully used for flower classification so far. However, given the developments of the past decade, many more methods were developed and successfully used, e.g., for general object or scene classification tasks. In this section we review and shortly explain prominent methods for the same steps of the processing pipeline.
In the previous section, seven research questions (RQ 1–7) were defined. These questions are answered by evaluating the introduced methods on three datasets: the Oxford Flower 17 (OF17), , the Oxford Flower 102 (OF102) , and our own Jena Flowers 30 (JF30) . Next, we describe these datasets.
We performed a comprehensive comparison of state-of-the-art methods within an image classification pipeline for flower image based plant species classification using local features. Hence, we investigated methods relevant for local feature detection, descriptor extraction, encoding, pooling, and fusion. We investigated the impact of the selected methods measured in terms of classification accuracy on three different datasets: the Oxford Flower 17, the Oxford Flower 102, as well as our own Jena Flower 30.