We present a theoretical framework that describes a wave packet of light prepared in a state of definite photon number interacting with an arbitrary quantum system (e.g. a quantum harmonic oscillator or a multi-level atom). Within this framework we derive master equations for the system as well as for output field quantities such as quadratures and photon flux. These results are then generalized to wave packets with arbitrary spectral distribution functions. Finally, we obtain master equations and output field quantities for systems interacting with wave packets in multiple spatial and/or polarization modes.