In a device-to-device (D2D) underlaid cellular network, the uplink spectrum is reused by the D2D transmissions, causing mutual interference with the ongoing cellular transmissions. Massive MIMO is appealing in such a context as the base station's (BS's) large antenna array can nearly null the D2D-to-BS interference. The multi-user transmission in massive MIMO, however, may lead to increased cellular-to-D2D interference. This paper studies the interplay between massive MIMO and underlaid D2D networking in a multi-cell setting. We investigate cellular and D2D spectral efficiency under both perfect and imperfect channel state information (CSI) at the receivers that employ partial zero-forcing. Compared to the case without D2D, there is a loss in cellular spectral efficiency due to D2D underlay. With perfect CSI, the loss can be completely overcome if the number of canceled D2D interfering signals is scaled with the number of BS antennas at an arbitrarily slow rate. With imperfect CSI, in addition to pilot contamination, a new asymptotic effect termed underlay contamination arises. In the non-asymptotic regime, simple analytical lower bounds are derived for both the cellular and D2D spectral efficiency.