Future wireless communication systems are envisioned to share radio frequency (RF)
spectrum, with other services such as radars, in order to meet the growing spectrum
demands. In this paper, we consider co-channel spectrum sharing between cellular systems
and radars. We address the problem of target detection by radars that are subject
to shape its waveform in a way that it does not cause interference to cellular systems.
We consider a multiple-input multiple-output (MIMO) radar and a MIMO cellular communication
system with \(\mc K\) base stations (BS). We propose a spectrum sharing algorithm which
steers radar nulls, by projecting radar waveform onto the null space of interference
channel, towards a `selected' BS, thus, protecting it from radar interference. This
BS is selected, among \(\mc K\) BSs, on the basis of guaranteeing minimum waveform degradation.
We study target detection capabilities of this null-space projected (NSP) waveform
and compare it with the orthogonal waveform. We derive the generalized likelihood
ratio test (GLRT) for target detection and derive detector statistic for NSP and orthogonal
waveform. The target detection performance for NSP and orthogonal waveform is studied
theoretically and via Monte Carlo simulations.