The 2016 QSM Challenge: Lessons learned and considerations for a future challenge design

Abstract

Purpose:

The 4 ^th International Workshop on MRI Phase Contrast & QSM (2016, Graz) hosted the first QSM challenge. A single-orientation GRE acquisition was provided, along with COSMOS and the χ ₃₃ STI component as ground-truths. The submitted solutions differed more than expected depending on the error metric used for optimization and tended to over-regularization. This raised (unanswered) questions about the ground-truths and the metrics utilized.

Methods:

We investigated the influence of background field remnants by applying additional filters. We also estimated the anisotropic contributions from the STI tensor to the apparent susceptibility, to amend the χ ₃₃ ground-truth and to investigate the impact on the reconstructions. Lastly, we used forward simulations from the COSMOS reconstruction to investigate the impact that has noise on the metrics.

Results:

Reconstructions compared against the amended STI ground-truth returned lower errors. We show that the background field remnants had a minor impact in the errors. In the absence of inconsistencies, all metrics converged to the same regularization weights, whereas SSIM was more insensitive to such inconsistencies.

Conclusion:

There was a mismatch between the provided data and the ground-truths due to the presence of unaccounted anisotropic susceptibility contributions and noise. Given the lack of reliable ground-truths when using in vivo acquisitions, simulations are suggested for future QSM challenges.