Multiplex networks are receiving increasing interests because they allow to model relationships between networked agents on several layers simultaneously. In this supplementary material for the paper "Navigability of interconnected networks under random failures", we extend well-known random walks to multiplexes and we introduce a new type of walk that can exist only in multiplexes. We derive exact expressions for vertex occupation time and the coverage. Finally, we show how the efficiency in exploring the multiplex critically depends on the underlying topology of layers, the weight of their inter-connections and the strategy adopted to walk.