Correlations of galaxy ellipticities with large-scale structure, due to galactic tidal interactions, provide a potentially significant contaminant to measurements of cosmic shear. However, these intrinsic alignments are still poorly understood for galaxies at the redshifts typically used in cosmic shear analyses. For spiral galaxies, it is thought that tidal torquing is significant in determining alignments resulting in zero correlation between the intrinsic ellipticity and the gravitational potential in linear theory. Here, we calculate the leading-order correction to this result in the tidal-torque model from non-linear evolution, using second-order perturbation theory, and relate this to the contamination from intrinsic alignments to the recently-measured cross-correlation between galaxy ellipticities and the CMB lensing potential. We find that the angular cross-correlation from tidal torquing has a very similar scale dependence as in the linear alignment model (believed to be appropriate for elliptical galaxies), but the opposite sign and so increases the observable correlation between CMB lensing and spiral galaxies. The amplitude of the cross-correlation is predicted to depend strongly on the formation redshift, being smaller for galaxies that formed at higher redshift when the bispectrum of the gravitational potential was smaller. Finally, we make simple forecasts for constraints on intrinsic alignments from the correlation of forthcoming cosmic shear measurements with current CMB lensing measurements.