We investigate the lattice \({\mathbb C} P^{N-1}\) sigma model on \(S_{s}^{1}\)(large) \(\times\) \(S_{\tau}^{1}\)(small) with the \({\mathbb Z}_{N}\) symmetric twisted boundary condition, where a sufficiently large ratio of the circumferences (\(L_{s}\gg L_{\tau}\)) is taken to approximate \({\mathbb R} \times S^1\). We find that the expectation value of the Polyakov loop, which is an order parameter of the \({\mathbb Z}_N\) symmetry, remains consistent with zero (\(|\langle P\rangle|\sim 0\)) from small to relatively large inverse coupling \(\beta\) (from large to small \(L_{\tau}\)). As \(\beta\) increases, the distribution of the Polyakov loop on the complex plane, which concentrates around the origin for small \(\beta\), isotropically spreads and forms a regular \(N\)-sided-polygon shape (e.g. pentagon for \(N=5\)), leading to \(|\langle P\rangle| \sim 0\). By investigating the dependence of the Polyakov loop on \(S_{s}^{1}\) direction, we also verify the existence of fractional instantons and bions, which cause tunneling transition between the classical \(N\) vacua and stabilize the \({\mathbb Z}_{N}\) symmetry. Even for quite high \(\beta\), we find that a regular-polygon shape of the Polyakov-loop distribution, even if it is broken, tends to be restored and \(|\langle P\rangle|\) gets smaller as the number of samples increases. To discuss the adiabatic continuity of the vacuum structure from another viewpoint, we calculate the \(\beta\) dependence of ``pseudo-entropy" density \(\propto\langle T_{xx}-T_{\tau\tau}\rangle\). The result is consistent with the absence of a phase transition between large and small \(\beta\) regions.