Effective designs compel customers to purchase products. Adding extra value to products by characterizing users’ experiences can lead firms to greater market success. Haptic stimulation can be added to products to enhance product performance and consumer satisfaction. Handheld products not only integrate sensors, such as pressure sensors and fingerprint scanners, into their buttons but also feature materials, shapes, and physical feedback, all of which enrich users’ haptic experience. In contrast to other sensory modalities, haptic sensors can detect various information types, such as pressure, weight, shape, and temperature, which are typically input in a single action. Products may induce various emotional responses depending on their shape and haptic feedback capabilities. Most studies have focused on the visual consistency of stimuli, but not semantic interference, between haptic modalities. This study investigated the relationship between the product semantic and affective response through cross-modal stimulation of button design. The results indicate that buttons with convex features result in high levels of arousal but low levels of valence, whereas buttons with an engraved square line result in the highest levels of valence. In the future, the event-related potential method will be used to test this study’s applicability.