While the construction of high-rise buildings is a popular policy strategy for accommodating population growth in cities, there is still much debate about the health consequences of living in high flats. This study examines the relationship between living in high-rise buildings and self-rated health in Belgium. We use data from the Belgian Census of 2001, merged with the National Register of Belgium ( N = 6,102,820). Results from multilevel, binary logistic regression analyses show that residents living in high-rise buildings have considerable lower odds to have a good or very good self-rated health in comparison with residents in low-rise buildings (OR 0.67; 95 % CI 0.67–0.68). However, this negative relationship disappears completely after adjusting for socioeconomic and demographic variables (OR 1.04; 95 % CI 1.03–1.05), which suggests that residents’ worse self-rated health in high-rise buildings can be explained by the strong demographic and socioeconomic segregation between high- and low-rise buildings in Belgium. In addition, there is a weak, but robust curvilinear relationship between floor level and self-rated health within high-rise buildings. Self-rated health increases until the sixth floor (OR 1.19; 95 % CI 1.15–1.24) and remains stable from the seventh floor and upwards. These findings refute one of the central ideas in architectural sciences that living in high buildings is bad for one’s health.