The indoor-outdoor connection provided by ventilation openings has been so far a limiting factor in the use of natural ventilation (NV), due to the apparent conflict between ventilation needs and the intrusion of external noise. This limiting factor impedes naturally ventilated buildings meeting the acoustic criteria set by standards and rating protocols, which are reviewed in this paper for residential buildings. The criteria reflect a general effort to minimize noise annoyance by reducing indoor sound levels, typically without a distinction based on a ventilation strategy. Research has developed a number of solutions, discussed here, that try to guarantee ventilation without compromising façade noise insulation, but, currently, none have been adopted on a large scale. This concept paper highlights the main limits of the current approach. First, a fragmented view towards indoor environmental quality has not included consideration of the following acoustic criteria: (i) how buildings are designed and operated to meet multiple needs other than acoustical ones (e.g., ventilation, visual, and cooling needs) and (ii) how people respond to multiple simultaneous environmental factors. Secondly, the lack of a perceptual perspective has led acoustic criteria to neglect the multiple cognitive and behavioral factors impinging on comfort in naturally ventilated houses. Indeed, factors such as the connection with the outside and the sense of control over one’s environment may induce “adaptive acoustic comfort” opportunities that are worth investigating. The mere use of different sound level limits would not be enough to define criteria tailored to the complex user–building interaction that occurs under NV conditions. More holistic and human-centered approaches are required to guarantee not only neutrally but even positively perceived indoor acoustic environments. For this reason, this paper considers this apparent conflict from a soundscape viewpoint, in order to expose still unexplored lines of research. By underpinning a perceptual perspective and by contextualizing it, the indoor soundscape approach provides a framework capable of overcoming the limits of the traditional noise control approach. This could provide the opportunity to foster a wider adoption of NV as a passive design strategy that enhances user health and well-being, while enabling low-cost, and low-energy cooling and ventilation, thereby contributing to current climate change challenges.