131
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Approximation method for discrete Markov decision models with a large state space

      Preprint

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          To solve discrete Markov decision models with a large number of dimensions is always difficult (and at times, impossible), because size of state space and computation cost increases exponentially with the number of dimensions. This phenomenon is called "The Curse of Dimensionality," and it prevents us from using models with many state variables. To overcome this problem, we propose a new approximation method, named statistical least square temporal difference (SLSTD) method, that can solve discrete Markov decision models with large state space. SLSTD method approximate the value function on the whole state space at once, and obtain optimal approximation weight by minimizing temporal residuals of the Bellman equation. Furthermore, a stochastic approximation method enables us to optimize the problem with low computational cost. SLSTD method can solve DMD models with large state space more easily than other existing methods, and in some cases, reduces the computation time by over 99 percent. We also show the parameter estimated by SLSTD is consistent and asymptotically normally distributed.

          Related collections

          Most cited references21

          • Record: found
          • Abstract: not found
          • Article: not found

          Learning to predict by the methods of temporal differences

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher

            John Rust (1987)
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              The Career Decisions of Young Men

                Bookmark

                Author and article information

                Journal
                1506.06722

                Mathematical modeling & Computation
                Mathematical modeling & Computation

                Comments

                Comment on this article