We present a study of the dielectric, structural, and magnetic properties of the multiferroic or linear magnetoelectric substitution series [(NH\(_4\))\(_{1-x}\)K\(_x\)]\(_2\)[FeCl\(_5\)(H\(_2\)O)]. Pyroelectric currents, magnetic susceptibilities, and thermodynamic properties were examined on large single crystals of the erythrosiderite compounds and detailed magnetic-field versus temperature phase diagrams are derived for three different substitution levels. With increasing potassium concentration the material is tuned from a multiferroic (\(x \le 0.06\)) to a linear magnetoelectric (\(x \ge 0.15\)) ground state. In contrast to the respective pure parent compounds with \(x=0\) or 1, however, the ferroelectric or linear magnetoelectric polarization in none of the substituted samples is switchable by external electric fields, because these samples exhibit a significant electric polarization already above the magnetic ordering transition. The polarization arises at a higher-lying structural phase transition that is examined by THz spectroscopy and, on a deuterated pure single crystal, by comprehensive neutron-diffraction experiments. The structural phase transition is attributed to an ordering of NH\(_4^+\) tetrahedra but does not break inversion symmetry in the pure material, while a finite K content causes pyroelectricity.