+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Deep learning‐based pose estimation for African ungulates in zoos


      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          1. The description and analysis of animal behavior over long periods of time is one of the most important challenges in ecology. However, most of these studies are limited due to the time and cost required by human observers. The collection of data via video recordings allows observation periods to be extended. However, their evaluation by human observers is very time‐consuming. Progress in automated evaluation, using suitable deep learning methods, seems to be a forward‐looking approach to analyze even large amounts of video data in an adequate time frame.

          2. In this study, we present a multistep convolutional neural network system for detecting three typical stances of African ungulates in zoo enclosures which works with high accuracy. An important aspect of our approach is the introduction of model averaging and postprocessing rules to make the system robust to outliers.

          3. Our trained system achieves an in‐domain classification accuracy of >0.92, which is improved to >0.96 by a postprocessing step. In addition, the whole system performs even well in an out‐of‐domain classification task with two unknown types, achieving an average accuracy of 0.93. We provide our system at https://github.com/Klimroth/Video‐Action‐Classifier‐for‐African‐Ungulates‐in‐Zoos/tree/main/mrcnn_based so that interested users can train their own models to classify images and conduct behavioral studies of wildlife.

          4. The use of a multistep convolutional neural network for fast and accurate classification of wildlife behavior facilitates the evaluation of large amounts of image data in ecological studies and reduces the effort of manual analysis of images to a high degree. Our system also shows that postprocessing rules are a suitable way to make species‐specific adjustments and substantially increase the accuracy of the description of single behavioral phases (number, duration). The results in the out‐of‐domain classification strongly suggest that our system is robust and achieves a high degree of accuracy even for new species, so that other settings (e.g., field studies) can be considered.


          We design and implement a video action classification system which is able to automatically detect three behavioral states of African ungulates. We achieve to overcome the challenges caused by bad video material (like huge amount of truncation, low frame rates, and night vision) which might be called the standard case in behavioral studies.

          Related collections

          Most cited references 52

          • Record: found
          • Abstract: not found
          • Article: not found

          ImageNet Large Scale Visual Recognition Challenge

            • Record: found
            • Abstract: not found
            • Conference Proceedings: not found

            Mask R-CNN

              • Record: found
              • Abstract: found
              • Article: not found

              JAABA: interactive machine learning for automatic annotation of animal behavior.

              We present a machine learning-based system for automatically computing interpretable, quantitative measures of animal behavior. Through our interactive system, users encode their intuition about behavior by annotating a small set of video frames. These manual labels are converted into classifiers that can automatically annotate behaviors in screen-scale data sets. Our general-purpose system can create a variety of accurate individual and social behavior classifiers for different organisms, including mice and adult and larval Drosophila.

                Author and article information

                Ecol Evol
                Ecol Evol
                Ecology and Evolution
                John Wiley and Sons Inc. (Hoboken )
                04 May 2021
                June 2021
                : 11
                : 11 ( doiID: 10.1002/ece3.v11.11 )
                : 6015-6032
                [ 1 ] Department of Computer Science and Mathematics Goethe University Frankfurt Germany
                [ 2 ] Faculty of Biological Sciences Bioscience Education and Zoo Biology Goethe University Frankfurt Germany
                Author notes
                [* ] Correspondence

                Max Hahn‐Klimroth, Department of Computer Science and Mathematics, Goethe University, 10 Robert Mayer St, Frankfurt 60325, Germany.

                Email: hahnklim@ 123456math.uni-frankfurt.de

                © 2021 The Authors. Ecology and Evolution published by John Wiley & Sons Ltd.

                This is an open access article under the terms of the http://creativecommons.org/licenses/by/4.0/ License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

                Page count
                Figures: 8, Tables: 7, Pages: 18, Words: 12883
                Funded by: von Opel Hessische Zoostiftung
                Original Research
                Original Research
                Custom metadata
                June 2021
                Converter:WILEY_ML3GV2_TO_JATSPMC version:6.0.2 mode:remove_FC converted:16.06.2021


                Comment on this article