Effects of interfacial roughness on the scattering of phonons at surfaces or solid/solid interfaces have been historically evaluated using a specularity parameter p formulated by Ziman [J. M. Ziman, Electrons and Phonons (Clarendon Press, Oxford, 1960)]. However, the validity of the Ziman theory being extended for boundary scattering of solid/solid interfaces has never been previously proven. In order to understand the interfacial scattering of thermal phonons and to test the validity of the Ziman theory, we accurately measure the in-plane thermal conductivity of a series of Si films in silicon-on-insulator (SOI) wafers by time-domain thermoreflectance (TDTR), for Si film thickness of 1 - 10 um and in a temperature range of 100 - 300 K. We carefully characterize the roughness of the Si/SiO2 interfaces in our SOI wafers as 0.11+/-0.04 nm using transmission electron microscopy (TEM). Furthermore, we build a relaxation time approximation (RTA) model using the mean-free-paths of phonons in Si crystal from first-principles calculations. We compare our in-plane thermal conductivity measurements to calculations of the RTA model and found that calculations using the specularity parameter based on Ziman's theory deviate significantly from the TDTR measurements. We attribute the discrepancy to the fact that Ziman's theory was derived for surfaces and does not take into consideration transmission of phonons at solid/solid interfaces. We thus derive a simple expression for the specularity parameter at solid/amorphous interfaces and achieve a good agreement between calculations using our expression and the measurements.