Remarks on measures of inconsistency

Abstract

Some inconsistencies between cosmological observations continue to persist. The currently reported inconsistencies can be due to problems with the underlying model or systematic effects associated with some particular datasets. In an era of precision cosmology, it is important to develop proper tools to quantify the degree of these inconsistencies and to look for methods to identify their causes. This turns out to be a task that is not free of ambiguities, especially when the model is multi-dimensional, which is usually the case in cosmology. Measures currently-proposed in the literature disagree even in Gaussian cases. We discuss, with illustrative examples, some requirements that should be fulfilled in Gaussian cases by inconsistency measures and suggest a guiding definition of (in)consistency. As an example, we show that the recently-proposed index of inconsistency (IOI) meets those requirements and is in line with the (in)consistency definition proposed. Next, we examine the common practice to convert some measures, including IOI and other similar quantities, to a probability to exceed or a significance level that depends on the number of parameters. In the context of quantifying inconsistencies, we show that such a procedure can underestimate inconsistencies when there is more than one model parameter. We also discuss multiple-dataset comparisons and introduce a new tool based on the multi-dataset IOI that can identify outlying constraints when present. Comparison of constraints from various datasets can help identify the source of inconsistencies and will become important as more independent constraints will become available from ongoing and future surveys.