Neurons in the hypothalamic arcuate nucleus (ARC) that co-express kisspeptin, neurokinin B and dynorphin (KNDy cells) are essential for mammalian reproduction as key regulators of gonadotropin-releasing hormone (GnRH) secretion. Although multiple endogenous and exogenous signals act indirectly via KNDy neurons to regulate GnRH, the identity of upstream neurons that provide synaptic input to this subpopulation is unclear. We used rabies-mediated tract-tracing in transgenic Kiss1-Cre mice combined with whole-brain optical clearing and multiple-label immunofluorescence to create a comprehensive and quantitative brain-wide map of neurons providing monosynaptic input to KNDy cells, as well as identify the estrogen receptor content and peptidergic phenotype of afferents. Over 90% of monosynaptic input to KNDy neurons originated from hypothalamic nuclei in both male and female mice. The greatest input arose from non-KNDy ARC neurons, including proopiomelanocortin-expressing cells. Significant female-dominant sex differences in afferent input were detected from estrogen-sensitive hypothalamic nuclei critical for reproductive endocrine function and sexual behavior in mice, indicating KNDy cells may provide a unique site for the coordination of sex-specific behavior and gonadotropin release. These data provide key insight into the structural framework underlying the ability of KNDy neurons to integrate endogenous and environmental signals important for the regulation of reproductive function.