Ultra-high performance concrete (UHPC) has superior mechanical properties and durability to normal strength concrete. However, the high amount of cement, high environmental impact, and initial cost are regarded as disadvantages, restricting its wider application. Incorporation of supplementary cementitious materials (SCMs) in UHPC is an effective way to reduce the amount of cement needed while contributing to the sustainability and cost. This paper investigates the mechanical properties and microstructure of UHPC containing fly ash (FA) and silica fume (SF) with the aim of contributing to this issue. The results indicate that, on the basis of 30% FA replacement, the incorporation of 10% and 20% SF showed equivalent or higher mechanical properties compared to the reference samples. The microstructure and pore volume of the UHPCs were also examined. Furthermore, to minimise the experimental workload of future studies, a prediction model is developed to predict the compressive strength of the UHPC using artificial neural networks (ANNs). The results indicate that the developed ANN model has high accuracy and can be used for the prediction of the compressive strength of UHPC with these SCMs.