Estrogen receptor alpha has a central role in human fertility by regulating estrogen action in all human reproductive tissues. Leukemia inhibitory factor (LIF) expression, a cytokine critical for blastocyst implantation, is mediated by estrogen signaling, so we hypothesized that ESR1 gene polymorphisms might be candidate risk markers for endometriosis-related infertility and in vitro fertilization (IVF) failure. We included 98 infertile women with endometriosis, 115 infertile women with at least one IVF failure and also 134 fertile women as controls. TaqMan SNP assays were used for genotyping LIF (rs929271), MDM2 (rs2279744), MDM4 (rs1563828), USP7 (rs1529916), and ESR1 (rs9340799 and rs2234693) polymorphisms. The SNP ESR1 rs9340799 was associated with endometriosis-related infertility ( P < 0.001) and also with IVF failure ( P = 0.018). After controlling for age, infertile women with ESR1 rs9340799 GG genotype presented 4-fold increased risk of endometriosis (OR 4.67, 95% CI 1.84–11.83, P = 0.001) and 3-fold increased risk of IVF failure (OR 3.33, 95% CI 1.38–8.03, P = 0.007). Our results demonstrate an association between ESR1 rs9340799 polymorphism and infertile women with endometriosis and also with women who were submitted to IVF procedures and had no blastocyst implantation.