Crohn’s Disease and Ulcerative Colitis are chronic, inflammatory conditions of the digestive tract, collectively known as Inflammatory Bowel Disease (IBD). The combined influence of lifestyle factors, genetics, and the gut microbiome contribute to IBD pathogenesis. Studies of the gut microbiome have shown significant differences in its composition between healthy individuals and those with IBD. Due to the high inter-individual microbiome variation seen in humans, mouse models of IBD are often used to investigate potential IBD mechanisms and their interplay between host, microbial, and environmental factors. While fecal samples are the predominant material used for microbial community analysis, they may not be the ideal sample to use for analysis of the microbiome of mice with experimental colitis, such as that induced by 2, 4, 6 trinitrobenzesulfonic acid (TNBS). As TNBS is administered intrarectally to induce colitis and inflammation is confined to the colon in this model, we hypothesized that the microbiome of the colonic mucus would most closely correlate with TNBS colitis severity. Based on our previous research, we also hypothesized that sex would be associated with both disease severity and microbial differences in mice with chronic TNBS colitis. We examined and compared the fecal, cecal content, and colonic mucus microbiota of 8-week old male and female C57BL/6J wild-type mice prior to and after the induction of TNBS colitis via 16S rRNA gene sequencing. We found that the colonic mucus microbiome was more closely correlated with disease severity than were alterations in the fecal and cecal microbiomes. We also found that the microbiomes of the feces, cecum, and mucus were distinct, but found no significant differences that were associated with sex in either compartment. Our findings highlight the importance of sampling colonic mucus in TNBS-induced colitis. Moreover, consideration of the differential impact of sex on the microbiome across mouse strains may be critical for the appropriate application of TNBS colitis models and robust comparisons across studies in the future.