Non-homogeneous random walks on a semi-infinite strip

Abstract

We study the asymptotic behaviour of Markov chains \((X_n,\eta_n)\) on \(\mathbb{Z}_+ \times S\), where \(\mathbb{Z}_+\) is the non-negative integers and \(S\) is a finite set. Neither coordinate is assumed to be Markov. We assume a moments bound on the jumps of \(X_n\), and that, roughly speaking, \(\eta_n\) is close to being Markov when \(X_n\) is large. This departure from much of the literature, which assumes that \(\eta_n\) is itself a Markov chain, enables us to probe precisely the recurrence phase transitions by assuming asymptotically zero drift for \(X_n\) given \(\eta_n\). We give a recurrence classification in terms of increment moment parameters for \(X_n\) and the stationary distribution for the large-\(X\) limit of \(\eta_n\). In the null case we also provide a weak convergence result, which demonstrates a form of asymptotic independence between \(X_n\) (rescaled) and \(\eta_n\). Our results can be seen as generalizations of Lamperti's results for non-homogeneous random walks on \(\mathbb{Z}_+\) (the case where \(S\) is a singleton). Motivation arises from modulated queues or processes with hidden variables where \(\eta_n\) tracks an internal state of the system.