Chinese summer extreme rainfall often brings huge economic losses, so the prediction of summer extreme rainfall is necessary. This study focuses on the predictability of the leading mode of Chinese summer extreme rainfall from empirical orthogonal function (EOF) analysis. The predictors used in this study are Arctic sea ice concentration (ASIC) and regional sea surface temperature (SST) in selected optimal time periods. The most important role that Arctic sea ice (ASI) plays in the appearance of EOF1 may be strengthening the high pressure over North China, thereby preventing water vapor from going north. The contribution of SST is mainly at low latitudes and characterized by a significant cyclone anomaly over South China. The forecast models using predictor ASIC (PA), SST (PS), and the two together (PAS) are established by using data from 1980 to 2004. An independent forecast is made for the last 11 years (2005–2015). The correlation coefficient (COR) skills between the observed and cross-validation reforecast principal components (PC) of the PA, PS, and PAS models are 0.47, 0.66, and 0.76, respectively. These values indicate that SST is a major cause of Chinese summer extreme rainfall during 1980–2004. The COR skill of the PA model during the independent forecast period of 2004–2015 is 0.7, which is significantly higher than those of the PS and PAS models. Thus, the main factor influencing Chinese summer extreme rainfall in recent years has changed from low latitudes to high latitudes. The impact of ASI on Chinese summer extreme rainfall is becoming increasingly significant.