10
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      A Lightweight Optical Flow CNN -- Revisiting Data Fidelity and Regularization

      Preprint
      , ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Over four decades, the majority addresses the problem of optical flow estimation using variational methods. With the advance of machine learning, some recent works have attempted to address the problem using convolutional neural network (CNN) and have showed promising results. FlowNet2, the state-of-the-art CNN, requires over 160M parameters to achieve accurate flow estimation. Our LiteFlowNet2 outperforms FlowNet2 on Sintel and KITTI benchmarks, while being 25.3 times smaller in the footprint and 3.1 times faster in the running speed. LiteFlowNet2 which is built on the foundation laid by conventional methods has marked a milestone to achieve the corresponding roles as data fidelity and regularization in variational methods. We present an effective flow inference approach at each pyramid level through a novel lightweight cascaded network. It provides high flow estimation accuracy through early correction with seamless incorporation of descriptor matching. A novel flow regularization layer is used to ameliorate the issue of outliers and vague flow boundaries through a novel feature-driven local convolution. Our network also owns an effective structure for pyramidal feature extraction and embraces feature warping rather than image warping as practiced in FlowNet2. Comparing to our earlier work, LiteFlowNet2 improves the optical flow accuracy on Sintel clean pass by 24%, Sintel final pass by 8.9%, KITTI 2012 by 16.8%, and KITTI 2015 by 17.5%. Our network protocol and trained models will be made publicly available on https://github.com/twhui/LiteFlowNet2 .

          Related collections

          Most cited references8

          • Record: found
          • Abstract: not found
          • Book Chapter: not found

          High Accuracy Optical Flow Estimation Based on a Theory for Warping

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them

              Bookmark
              • Record: found
              • Abstract: not found
              • Conference Proceedings: not found

              Optical Flow Estimation Using a Spatial Pyramid Network

                Bookmark

                Author and article information

                Journal
                15 March 2019
                Article
                1903.07414
                ea5e366f-bdba-4546-a794-2639fc024b6c

                http://creativecommons.org/licenses/by-nc-sa/4.0/

                History
                Custom metadata
                arXiv admin note: substantial text overlap with arXiv:1805.07036
                cs.CV

                Computer vision & Pattern recognition
                Computer vision & Pattern recognition

                Comments

                Comment on this article