Dietary patterns are linked to obesity, but the gender difference in the association between dietary patterns and obesity remains unclear. We explored this gender difference in a middle-aged and elderly populations in Shanghai. Residents ( n = 2046; aged ≥45 years; 968 men and 1078 women) who participated in the Shanghai Food Consumption Survey were studied. Factor analysis of data from four periods of 24-h dietary recalls (across 2012–2014) identified dietary patterns. Height, body weight, and waist circumference were measured to calculate the body mass index. A log binominal model examined the association between dietary patterns and obesity, stratified by gender. Four dietary patterns were identified for both genders: rice staple, wheat staple, snacks, and prudent patterns. The rice staple pattern was associated positively with abdominal obesity in men (prevalence ratio (PR) = 1.358; 95% confidence interval (CI) 1.132–1.639; p = 0.001), but was associated negatively with general obesity in women (PR = 0.745; 95% CI: 0.673–0.807; p = 0.031). Men in the highest quartile of the wheat staple pattern had significantly greater risk of central obesity (PR = 1.331; 95% CI: 1.094–1.627; p = 0.005). There may be gender differences in the association between dietary patterns and obesity in middle-aged and elderly populations in Shanghai, China.