In the last few years, increasing attention has been provided to research Vehicle-to-Pedestrian (V2P) communication systems. These V2P systems serve different purposes (safety or convenience) and cater to different Vulnerable Road User (VRU) groups. Also, these V2P systems employ different communication technologies, and use different mechanisms to interact with the users. An effective V2P system also needs to consider varying characteristics of different VRUs. These various elements may be considered as design parameters of the V2P system. In this paper, we discuss such elements and propose a design framework for the V2P system based on them. We also provide an extensive survey of existing V2P efforts for safety and convenience applications and their design considerations. We perform a case study that compares the different approaches of V2P safety system for different VRU groups under different pre-crash scenarios. Finally, we discuss a few technological challenges in integration of VRUs into V2X systems.