The prevalence of antibiotic resistance genes (ARGs) and their relationship with antibiotics in the surface waters of the Huangpu River and the drinking water sources of Shanghai, China, were investigated. 39 ARGs, including four sulfonamide ARGs, 23 tetracycline ARGs, four chloramphenicol ARGs, five β-lactam ARGs and three penicillin ARGs were targeted in this study. Two sulfonamide ARGs (sul I and sul II), eight tetracycline ARGs (tet(A), tet(B), tet(C), tet(G), tet(M), tet(O), tet(W) and tet(X)), and one β-lactam ARG (TEM) were detected to be present in water samples by polymerase chain reaction (PCR), with the detection frequencies ranging from 42.86% to 100%. The average concentrations of the 11 ARGs ranged from 3.66×10(1) copy/mL (tet(B)) to 1.62×10(5) copy/mL (sul II), quantified by quantitative real-time PCR (QPCR). The number of detected ARGs and the ARG concentrations were generally higher in suburban sampling sites than in urban sites. Being in or near suburban rural areas, some raw drinking water sources were observed to have comparatively higher ARG contamination, drawing an urgent attention for the concern of public health. Generally consistent relations were observed between the concentrations of tet genes and the tetracycline levels and between the concentrations of sul genes and the sulfonamide levels.