Dielectric polymers with high dipole density have the potential to achieve very high energy density, which is required in many modern electronics and electric systems. We demonstrate that a very high energy density with fast discharge speed and low loss can be obtained in defect-modified poly(vinylidene fluoride) polymers. This is achieved by combining nonpolar and polar molecular structural changes of the polymer with the proper dielectric constants, to avoid the electric displacement saturation at electric fields well below the breakdown field. The results indicate that a very high dielectric constant may not be desirable to reach a very high energy density.