Social wasps commonly exhibit impressive, pheromone-mediated nest defenses with stinging attacks on potential vertebrate nest predators. Studying this type of nest defense and comparing results across studies is challenging because there is no standardized method for quantifying defense intensities. For that reason, we developed a simple, paired-target apparatus coupled with easy and inexpensive data recording and analysis technologies. Each target is formed by two conjoined black plastic weigh boats that generate distinct percussive sounds when struck by attacking wasps. A battery-powered microphone inside each target converts the sounds into electrical signals that are transferred to a digital audio recorder. These audio files are then split into left- and right-channel files, saved as 16-bit WAV files, and the strikes to each target are counted using the open-source software SoundRuler. Using this apparatus, we show that workers of Vespula pensylvanica, V. alascensis, and V. germanica strike targets that are treated with conspecific venom sac extract more frequently than paired control targets. We also show that workers of V. alascensis, V. pensylvanica and V. germanica strike targets that are treated with heterospecific extracts more frequently than paired control targets, indicating that the wasps recognize nest alarm pheromones from congeners. These data provide evidence for conserved nest defense pheromones among some Vespula wasps and proof of concept that our technology is capable of quantifying the intensity of pheromone-mediated nest defense behavior in Vespula and other large and formidable social wasps.