Poisson algebras are, just like Lie algebras, particular cases of Lie-Rinehart algebras. The latter were introduced by Rinehart in his seminal 1963 paper, where he also introduces the notion of an enveloping algebra and proves --- under some mild conditions --- that the enveloping algebra of a Lie-Rinehart algebra satisfies a Poincar\'e-Birkhoff-Witt theorem (PBW theorem). In the case of a Poisson algebra \(({\mathcal A},\cdot,\{\cdot,\cdot\})\) over a commutative ring \(R\) (with unit), Rinehart's result boils down to the statement that if \(\mathcal A\) is \emph{smooth} (as an algebra), then gr\((U({\mathcal A}))\) and \(\mathrm{Sym}_{\mathcal A}(\Omega({\mathcal A}))\) are isomorphic as graded algebras; in this formula, \(U({\mathcal A})\) stands for the Poisson enveloping algebra of \({\mathcal A}\) and \(\Omega({\mathcal A})\) is the \({\mathcal A}\)-module of K\"ahler differentials of \({\mathcal A}\) (viewing \({\mathcal A}\) as an \(R\)-algebra). In this paper, we give several new constructions of the Poisson enveloping algebra in some general and in some particular contexts. Moreover, we show that for an important class of \emph{singular} Poisson algebras, the PBW theorem still holds. In geometrical terms, these Poisson algebras correspond to (singular) Poisson hypersurfaces of arbitrary smooth affine Poisson varieties.