In the current study, the overall prevalence and the main underlying etiologies of inflammatory low back pain (ILBP) were determined, and the effectiveness of treatment with sulfasalazine was investigated in patients with inflammatory versus mechanical low back pain (LBP). In a prospective study conducted from July 2013 until August 2015, 1,779 consecutive patients within the age range of 18–50 years with a primary complaint of LBP referring to the rheumatology clinics were included. The patients were classified into two distinct groups: those suffering from ILBP (n=118) and those having mechanical LBP (n=1,661). Patients were followed-up for assessing the response rate to sulfasalazine with a mean follow-up time of 16 months. Results showed that among the total number of participants, 6.6% suffered from ILBP. The main underlying diagnoses of ILBP were undifferentiated spondyloarthropathy (USpA) (61.0%) and ankylosing spondylitis (24.6%). During the follow-up period, 3.4% of the participants had an appropriate response to only nonsteroidal anti-inflammatory drugs, 57.6% to sulfasalazine, 26.3% to addition of methotrexate to the previous regimen, and 12.7% to biological agents. Multiple logistic regression results showed that the underlying disease had a significant effect on the sulfasalazine response. The odds for response to treatment was 3.53 times higher in USpA patients compared to other patients (odds ratio =3.53, 95% confidence interval: 1.63–7.68, P=0.001). In 69.4% of the participants, the highest response to sulfasalazine was found, which was related to the underlying USpA. This study found that an adequate response to nonsteroidal anti-inflammatory drugs in patients with ILBP was potentially increased by adding sulfasalazine. Thus, the observed response rate was dependent on the nature of underlying spondyloarthropathy.