Background Recent randomized trials have not found that polymyxin B hemoperfusion (PMX-HP) improves outcomes for patients with sepsis. However, it remains unclear whether the therapy could provide benefit for highly selected patients. Monocyte human leukocyte antigen (mHLA-DR) expression, a critical step in the immune response, is decreased during sepsis and leads to worsening sepsis outcomes. One recent study found that PMX-HP increased mHLA-DR expression while another found that the treatment removed HLA-DR-positive cells. Methods We conducted a randomized controlled trial in patients with blood endotoxin activity assay (EAA) level ≥ 0.6. Patients in the PMX-HP group received a 2-h PMX-HP treatment plus standard treatment for 2 consecutive days. Patients in the non-PMX-HP group received only standard treatment. The primary outcome compared the groups on median change in mHLA-DR expression between day 3 and baseline. Secondary outcomes compared the groups on the mean or median change in CD11b expression, neutrophil chemotaxis, presepsin, cardiovascular Sequential Organ Failure Assessment (CVS SOFA) score, vasopressor dose, and EAA level between day 3 and baseline. We further compared the groups on mortality, ICU-free days, ventilator-free days, dialysis dependence status, renal recovery, serum creatinine, vasopressor-free days, and major adverse kidney events (MAKE 28), measured on day 28. Results Fifty-nine patients were randomized to PMX-HP (n = 29) and non-PMX-HP (n = 30) groups. At baseline, mHLA-DR expression, CD11b, neutrophil chemotaxis, and clinical parameters were comparable between groups. The median change in mHLA-DR expression between day 3 and baseline was higher in PMX-HP patients than in patients receiving standard therapy alone (P = 0.027). The mean change in CD11b between day 3 and baseline was significantly lower in the PMX-HP group than in the non-PMX-HP group (P = 0.002). There were no significant changes from baseline in neutrophil chemotaxis, presepsin, CVS SOFA scores, vasopressor doses, or EAA level between groups. On day 28 after enrollment, mortality, ICU-free days, ventilator-free days, dialysis dependence status, renal recovery, serum creatinine, vasopressor-free days, and MAKE 28 were comparable between groups. Conclusion PMX-HP improved mHLA-DR expression in severe sepsis patients. Future studies should examine the potential benefit of PMX-HP in patients with low mHLA-DR expression. Trial registration ClinicalTrials.gov, NCT02413541. Registered on 3 March 2015. Electronic supplementary material The online version of this article (10.1186/s13054-018-2077-y) contains supplementary material, which is available to authorized users.