We investigate the modulation of optical phonons in semiconductor crystal by surface acoustic wave (SAW) propagating on the crystal surface. The SAW fields induce changes on the order of 10\textsuperscript{-3} in the average Raman scattering intensity by optical phonons in Si and GaN crystals. The SAW-induced modifications in the Raman cross-section are dominated by the modulation of the optical phonon energy by the SAW strain field. In addition to this local contribution, the experiments give evidence for a weaker and non-local contribution arising from the spatial variation of the SAW strain field. The latter is attributed to the activation of optical modes with large wave vectors and, therefore, lower energies. The experimental results, which are well described by theoretical models for the two contributions, prove that optical phonons can be manipulated by SAWs with \(\mu\)m wavelengths