2
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Count-Based Exploration in Feature Space for Reinforcement Learning

      Preprint

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          We introduce a new count-based optimistic exploration algorithm for Reinforcement Learning (RL) that is feasible in environments with high-dimensional state-action spaces. The success of RL algorithms in these domains depends crucially on generalisation from limited training experience. Function approximation techniques enable RL agents to generalise in order to estimate the value of unvisited states, but at present few methods enable generalisation regarding uncertainty. This has prevented the combination of scalable RL algorithms with efficient exploration strategies that drive the agent to reduce its uncertainty. We present a new method for computing a generalised state visit-count, which allows the agent to estimate the uncertainty associated with any state. Our \phi-pseudocount achieves generalisation by exploiting same feature representation of the state space that is used for value function approximation. States that have less frequently observed features are deemed more uncertain. The \phi-Exploration-Bonus algorithm rewards the agent for exploring in feature space rather than in the untransformed state space. The method is simpler and less computationally expensive than some previous proposals, and achieves near state-of-the-art results on high-dimensional RL benchmarks.

          Related collections

          Most cited references3

          • Record: found
          • Abstract: found
          • Article: not found

          Reinforcement Learning: An Introduction

          IEEE Transactions on Neural Networks, 9(5), 1054-1054
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            An analysis of model-based Interval Estimation for Markov Decision Processes

              Bookmark
              • Record: found
              • Abstract: not found
              • Conference Proceedings: not found

              Context Tree Switching

                Bookmark

                Author and article information

                Journal
                2017-06-25
                Article
                1706.08090
                7a52a3f0-542d-473b-a11a-5d5550e69749

                http://arxiv.org/licenses/nonexclusive-distrib/1.0/

                History
                Custom metadata
                Conference: Twenty-sixth International Joint Conference on Artificial Intelligence (IJCAI-17), 8 pages, 1 figure
                cs.AI

                Artificial intelligence
                Artificial intelligence

                Comments

                Comment on this article