The concentrations of rare earth elements (REEs) in the bulk sediment of Core X2, which was collected from southeastern Hainan Island, were analyzed to investigate the relative contributions of various provenance regions since mid-Holocene. The results show that sediments in Core X2 were primarily derived from Hainan Island with lesser amounts from Taiwan and limited input from the Pearl River. Based on the application of quantitative inversion to model the REE data, the average contributions of river materials from southeastern Hainan Island and southwestern Taiwan to the study area were 68% and 32%, respectively. Furthermore, starting at 4.0 kyr BP, the transport of fluvial sediments from Taiwan to the study region increased due to enhanced hydrodynamics in South China Sea (SCS). These results indicate that the contributions of mountain river materials from Hainan Island and Taiwan to the continental shelf of northern SCS are non-negligible. Furthermore, these results demonstrate that mountain rivers can play an important role in the material cycle of continental margins and may feature a greater impact than large river systems in specific continental shelf areas.