We re-investigate how generic off-diagonal cosmological solutions depending, in general, on all spacetime coordinates can be constructed in massive and f-modified gravity using the anholonomic frame deformation method. There are constructed new classes of locally anisotropic and (in) homogeneous cosmological metrics with open and closed spatial geometries. By resorting such solutions, we show that they describe the late time acceleration due to effective cosmological terms induced by nonlinear off-diagonal interactions, possible modifications of the gravitational action and graviton mass. The cosmological metrics and related St\" uckelberg fields are constructed in explicit form up to nonholonomic frame transforms of the Friedmann-Lama\^{\i}tre-Robertson-Walker (FLRW) coordinates. The solutions include matter, graviton mass and other effective sources modelling nonlinear gravitational and matter fields interactions with polarization of physical constants and deformations of metrics, which may explain dark energy and dark matter effects. However, we argue that it is not obligatory always to modify gravity if we consider effective generalized Einstein equations with nontrivial vacuum and/or non-minimal coupling with matter. Indeed, we state certain conditions when such configurations mimic interesting solutions in general relativity and modifications, for instance, when we can extract the general Painlev\' e-Gullstrand and FLRW metrics. In a more general context, we elaborate on a reconstruction procedure for off-diagonal cosmological solutions which describe cyclic and ekpyrotic universes. Finally, there are discussed open issues and further perspectives.