Dear Editor,
We read with interest the recently published meta-analysis report titled “Chicken
Essence for Cognitive Function Improvement: A Systematic Review and Meta-Analysis”
by Teoh et al. [1] in Nutrients. We feel that a substantial number of questions are
raised by this analysis, none of which are sufficiently addressed by the authors.
The questions and issues can be grouped into general categories, and these are given
below in order of decreasing importance.
1. Methodological and External Validity Issues
Reference 2 (Azhar et al., 2013 [2]) was a study on CMI-168 (as described in the abstract),
not Chicken Essence, and thus this study was inappropriate for inclusion.
Of the studies considered for inclusion, a very large number had to be eliminated.
While the grounds for inclusion in the combined analysis were very broad, of 2203
non-duplicate studies reviewed, only seven were found to be suitable for qualitative
synthesis and five for meta-analysis, and the total number of included subjects (363)
was relatively small. In four of these studies there was either an unacceptably high
dropout rate (>20%) or the dropout rate was not stated.
Fifty-six of the subjects came from a single study of patients with poor cognitive
function (Reference 14 (Azhar et al., 2003 [3]), while the remainder of the subjects
came from populations of healthy volunteers. In our view, combining cognitively impaired
subjects’ results with those of healthy volunteers is not valid; grouping these subjects
together introduces the potential for bias, as possible outliers may neutralize any
statistically or clinically significant results. In addition, five of the studies
used a narrow age range of young subjects (<40 years)—these subjects are less likely
to show a cognitive effect even if one existed.
The multiplicity of endpoints in the seven studies made it impractical to measure
anything consistently across all the studies. For example, Azhar et al. [2] used in
the meta-analysis measured long-term memory, while in the other studies, aspects of
short-term memory, working memory, attention, etc., were measured. Study designs also
differed, as four of the studies were parallel and three were crossover.
The product types, formats, treatment periods, and dosages were extremely heterogeneous
across the seven studies.
The authors attempted to establish a connection between Chicken Essence, which includes
a wide variety of peptides, and cognitive function. In the Discussion section, the
authors compare the results of Chicken Essence studies to studies using carnosine.
While carnosine is one of the many peptides that can be found in the Chicken Essence,
the Chicken Essence products contain hundreds of different peptides with various activity
levels; in addition, carnosine makes up less than 1/1000 fraction of Chicken Essence
solid content. Because of these factors and the differing formulations of products
used in the studies, the effect of carnosine on cognitive function and its mechanism
of action cannot be determined. Therefore, there is no rationale to link Chicken Essence
studies to carnosine studies as the authors have done.
“Checking” methods (e.g., Cochran’s Risk of Bias) applied to the various studies determined
that the individual study results were, a priori, unreliable. We believe that it does
not make sense to combine a small group of poorly done studies in a meta-analysis.
2. Statistical Issues
Thirty-five significance tests of mean differences were conducted and twelve were
significant at the 0.05 level. While the significant outcomes tended to group together
(in Executive Function and Short Term Memory, as one would expect if these assessments
are intercorrelated), the actual tests were mostly different in the different studies,
making comparisons difficult.
3. Accuracy Issues
Table 4 identifies Executive Function, WAIS arithmetic test, Azhar et al., 2008 [4],
as significant, but the 95% CI includes zero.
4. Conclusions
While the authors concluded that a claim of cognitive benefit for Chicken Essence
was not yet supported by published data, the very large number of methodological issues
identified above leads us to believe that, at this point in time, evidence produced
in this paper are too scanty and potentially unreliable to permit any conclusion,
either positive or negative, to be drawn. To assess the impact of Chicken Essence
on cognitive function through a meta-analysis, future research needs to identify studies
that are consistent across product formulation, dosage, and treatment duration, use
subjects in sufficient numbers drawn from comparable populations, and have clear,
homogeneous cognitive endpoints.