DeepAction: A MATLAB toolbox for automated classification of animal behavior in video

Abstract

The identification of behavior in video is a critical but time-consuming component in many areas of animal behavior research. Here, we introduce DeepAction, a deep learning-based toolbox for automatically annotating animal behavior in video. Our approach uses features extracted from raw video frames by a pretrained convolutional neural network to train a recurrent neural network classifier. We evaluate the classifier on two benchmark rodent datasets and show that it achieves high accuracy, requires little training data, and surpasses both human agreement and similar existing methods. We also create a confidence score for classifier output, and show our method provides an accurate estimate of classifier performance and reduces the time required by human annotators to review and correct automatically-produced annotations. We release our system and accompanying annotation interface as an adaptable, non-technical, and open-source MATLAB toolbox.