We compare the 230 GHz near-horizon emission from Sagittarius A* to simulations representing three classes of accretion flows. Using the structure function to capture the variability statistics of the light curve, we find a noticeable discrepancy between the observations and models based on torus-fed accretion disks, whether those disks bring in a small or large amount of net magnetic flux. On the other hand, the simulations that are fed more realistically by stellar winds match the observed structure function very well. We describe the differences between models, arguing that feeding by stellar winds may be a critical component in constructing theoretical models for accretion in the Galactic Center.