We analyzed the rapid aperiodic X-ray variability of different types of X-ray binaries (black hole candidates, atoll sources, the recently discovered millisecond X-ray pulsar, and Z sources) at their lowest inferred mass accretion rates. At these accretion rates, the power spectra of all sources are dominated by a strong band-limited noise component, which follows a power law with an index roughly 1 at high frequencies and breaks at a frequency between 0.02 and 32 Hz below which the spectrum is relatively flat. Superimposed on this, a broad bump (sometimes a quasi-periodic oscillation) is present with a 0.2-67 Hz centroid frequency that varies in good correlation with the frequency of the break. The black hole candidates and the low-luminosity neutron star systems (including the millisecond X-ray pulsar) have the same relation between the frequency of the bump and the frequency of the break. These similar characteristics strongly suggest that in all those different types of sources the band-limited noise and the bump are produced by the same physical mechanism. This mechanism cannot then depend on the presence or absence of either a small magnetosphere or a solid surface, so that it is most likely related to an instability in the flow in the accretion disk that modulates the accretion rate. The Z sources, which are more luminous than the other sources discussed here, follow a similar, but slightly shifted correlation between the break frequency and the frequency of the bump. The data suggest that the band-limited noise in Z sources is more complex than that in the other sources.