We use the Illustris simulation to study the relative contributions of in situ star formation and stellar accretion to the build-up of galaxies over an unprecedentedly wide range of masses (\(M_{\ast} = 10^9-10^{12} \, {\rm M_{\odot}}\)), galaxy types, environments, and assembly histories. We find that the `two-phase' picture of galaxy formation predicted by some models is a good approximation only for the most massive galaxies in our simulation -- namely, the stellar mass growth of galaxies below a few times \(10^{11} \, {\rm M_{\odot}}\) is dominated by in situ star formation at all redshifts, while galaxies above this mass at \(z \lesssim 1\) grow primarily by accretion of stars via mergers. The fraction of the total stellar mass of galaxies at \(z=0\) contributed by accreted stars shows a strong dependence on galaxy stellar mass, ranging from about 10% for Milky Way-sized galaxies to over 80% for \(M_{\ast} \approx 10^{12} \, {\rm M_{\odot}}\) objects, yet with a large galaxy-to-galaxy variation. At a fixed stellar mass, elliptical galaxies and those formed at the centres of younger haloes exhibit larger fractions of ex situ stars than disc-like galaxies and those formed in older haloes. On average, \(\sim\)50% of the ex situ stellar mass comes from major mergers (stellar mass ratio \(\mu > 1/4\)), \(\sim\)20% from minor mergers (\(1/10 < \mu < 1/4\)), \(\sim\)20% from very minor mergers (\(\mu < 1/10\)), and \(\sim\)10% from stars that were stripped from surviving galaxies (e.g. flybys or ongoing mergers). These components are spatially segregated, with in situ stars dominating the innermost regions of galaxies, and ex situ stars being deposited at larger galactocentric distances in order of decreasing merger mass ratio. The `transition' radius where ex situ stars begin to dominate over the in situ class decreases for more massive galaxies and correlates strongly with the total ex situ fraction.