We prove a global existence result with initial data of low regularity, and prove the trend to the equilibrium for the Vlasov-Poisson-Fokker-Planck system with small non linear term but with a possibly large exterior confining potential in dimension \(d=2\) and \(d=3\). The proof relies on a fixed point argument using sharp estimates (at short and long time scales) of the semi-group associated to the Fokker-Planck operator, which were obtained by the first author.