The Computation of the Mean First Passage Times for Markov Chains

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

A survey of a variety of computational procedures for finding the mean first passage times in Markov chains is presented. The author recently developed a new accurate computational technique, an Extended GTH Procedure, Hunter (Special Matrices, 2016) similar to that developed by Kohlas (Zeit. fur Oper. Res., 1986). In addition, the author has recently developed a variety of new perturbation techniques for finding key properties of Markov chains including finding the mean first passage times, Hunter (Linear Algebra and its Applications, 2016). These recently developed procedures are compared with other procedures including the standard matrix inversion technique using the fundamental matrix (Kemeny and Snell, 1960), some simple generalized matrix inverse techniques developed by Hunter (Asia Pacific J. Oper. Res., 2007), and the FUND technique (with some modifications) of Heyman (SIAM J Matrix Anal. and Appl., 1995). MatLab is used to compute errors when the techniques are used on some test problems that have been used in the literature. A preference for the accurate procedure of the author is exhibited.

Related collections

Most cited references 8

Record: found
Abstract: not found
Article: not found

Generalized inverses and their application to applied probability problems

Jeffrey Hunter (1982)

0 comments Cited 14 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

On the moments of Markov renewal processes

Jeffrey Hunter (1969)

Recently Kshirsagar and Gupta [5] obtained expressions for the asymptotic values of the first two moments of a Markov renewal process. The method they employed involved formal inversion of matrices of Laplace-Stieltjes transforms. Their method also required the imposition of a non-singularity condition. In this paper we derive the asymptotic values using known renewal theoretic results. This method of approach utilises the fundamental matrix of the imbedded ergodic Markov chain and the theory of generalised matrix inverses. Although our results differ in form from those obtained by Kshirsagar and Gupta [5] we show that they reduce to their results under the added non-singularity condition. As a by-product of the derivation we find explicit expressions for the moments of the first passage time distributions in the associated semi-Markov process, generalising the results of Kemeny and Snell [4] obtained for Markov chains.