Why meta is better: A reply to Linden‐Carmichael et al. (2018)

Abstract

Linden‐Carmichael et al. (this issue) rightly caution against prematurely drawing conclusions regarding the safety of consuming alcohol mixed with energy drinks (AMED). Our aim in conducting a meta‐analysis (Verster et al., 2018) was not to play down the risks associated with AMED consumption, rather it was to give an objective picture of the literature on functional consequences of AMED consumption using data available at the time of conducting the review. Linden‐Carmichael et al. suggest that their recent diary method paper (Linden‐Carmichael & Lau‐Barraco, 2017) casts doubt on the conclusions of our meta‐analysis. As the paper in question was, in the authors' words, “published after Verster and colleagues completed their literature review,” it was not included. Nevertheless, the paper is a welcome contribution to the AMED literature. To suggest, however, that this single paper supersedes all previous work in the field does seem rather premature. Consuming alcohol in any form, including AMED, carries risks. It is important, however, that the functional consequences of AMED and alcohol only (AO) consumption are reported on the basis of empirical, evidence‐based analyses. Our meta‐analysis paper confirms that, compared with AO consumers, AMED users consume more alcohol and are subject to more alcohol‐related harms. It is also true that AMED consumers differ significantly from non‐AMED consumers on a number of other dimensions, e.g., they are significantly more likely to be male, taller, and use drugs (e.g., De Haan et al., 2012). We have therefore cautiously suggested that AMED consumption may be, at least partly, one of several phenotypical manifestations of some other dispositional factor or trait. One of the advantages of meta‐analysis is that the method captures and synthesizes the extant literature. In this case, as a secondary aim, we can examine whether the results of the meta‐analysis are consistent with our hypothesis by focusing on any differences in between‐subjects and within‐subjects comparisons. AMED consumers tend to mix alcohol with energy drinks on a minority of drinking occasions (Verster et al., 2018; Verster, Aufricht, & Alford, 2012). If AMED consumption is one of several manifestations of some underlying trait, we would predict that AMED consumers would drink a similar amount of alcohol (and have a similar frequency of alcohol‐related consequences) when they drink AMED to when they consume AO. Alternatively, If AMED consumption is causal in producing alcohol‐related harms, then one would predict that AMED users would drink more (and be subject to more negative alcohol‐related consequences) on AMED occasions than on AO occasions. Our meta‐analysis (Verster et al., 2018) shows unequivocally that (a) compared with those who never consume AMED but consume AO, AMED consumers drink more alcohol and, by extension, engage in more alcohol‐related harmful behaviors (between‐subjects analysis of N = 6,061 AMED and N = 14,496 AO consumers in total); (b) compared with when they consume AO, on the occasions where AMED drinkers co‐consume alcohol and energy drinks, they do not consume more alcohol and, by the same argument, not engage in more alcohol‐related harmful behaviors (within‐subjects analysis of N = 3,480 AMED consumers). This result, which would hold irrespective of who conducted the meta‐analysis, supports the notion that AMED consumption may be, at least in part, one manifestation of an underlying trait. This is important because better understanding of the reasons why certain individuals become AMED consumers (possibly as one of a cluster of harmful drug and alcohol related behaviors) may help in the early identification of these individuals in order to instigate preventative measures. An alternative view is that AMED consumption is causal in producing alcohol‐related harms, so a better strategy would be to differentially restrict access to AMED over other forms of alcohol. The meta‐analysis supports the former. This does not preclude the possibility of other processes, not examined in the meta‐analysis, being differentially associated with AMED; nor does it mean there should be no further investigation into the possible harms associated with AMED. Indeed, as we state in the abstract of our paper, “Further research may be necessary to fully reveal the effects of AMED.” (Verster et al., 2018). Linden‐Carmichael et al. (this issue) identify possible problems with survey methods, in particular that they can be subject to recall bias—presumably with this recall bias differentially affecting memory of AMED over AO occasions. Specifically, this would require recall bias to result in under‐reporting of harms during AMED but not AO occasions. No mechanism is put forward for this differential effect, although it may be that, as AMED consumers tend to drink AO on the majority of occasions, there is more opportunity to recall AO‐related harms. It is certainly true that such studies are imperfect, but they can provide useful information for meta‐analyses and reviews. For example, the same authors' 2014 “qualitative review of psychosocial risk factors” associated with AMED use concludes that “use of such beverages is associated with negative consequences including heavy alcohol use, risky sexual and driving behaviors, as well as other drug use” (Linden & Lau‐Barraco, 2014). Notably, when rallying around 20 surveys to support their contention, the authors did not once mention the possibility of recall bias. We concur with Linden‐Carmichael et al. (this issue) that AMED research presents its own unique problems. Unfortunately, this research area has been undermined to some degree by selective reporting of the harms associated with AMED. However, we believe that meta‐analyses can provide useful information that may be complemented and challenged by single studies but not undermined by them. As an example, Linden‐Carmichael and Lau‐Barraco (2017) suggest that “one reason for the link between CAB use and alcohol outcomes may be that caffeine can reduce one's feelings of intoxication without reducing actual drunkenness” (note that CAB refers to caffeinated alcoholic beverage, including AMED). The phenomenon of “masking”, the notion that coconsuming caffeine with alcohol could reduce perceived intoxication while leaving alcohol impairment unaffected is not supported by the literature. Rather than reference a meta‐analysis (Benson, Verster, Alford, & Scholey, 2014), which included all subjective intoxication studies, Linden‐Carmichael and Lau‐Barraco (2017) cite a single study by Marczinski and Fillmore (2006) to support this contention. They report that Marczinski and Fillmore “found after consuming CABs as opposed to regular alcohol, participants felt less intoxicated” (p. 882). In fact, despite an abstract concluding that “subjective measures of intoxication showed that coadministration of caffeine with alcohol reduced participants' perceptions of alcohol intoxication compared with administration of alcohol alone” (Marczinski & Fillmore, 2006), this finding is not as clear as stated. The study reported that a lower (2 mg/kg) dose, equivalent to an average of around 140 mg of caffeine in their sample, reduced self‐rated alcohol intoxication, whereas a higher (4 mg/kg or 280 mg) dose of caffeine did not. Further, Marczinski and Fillmore state that “coadministration of 2.0 mg/kg of caffeine with alcohol significantly lowered beverage ratings [a measure of subjective intoxication] as compared with alcohol alone, t(11) = 1.77, p = .05” (p. 455). Taken at face value, a lower but not a higher dose of caffeine had, at most, a marginally significant effect on perceived intoxication. The picture is further complicated, however, by a statistical anomaly in the Marczinski and Fillmore (2006) paper, which has previously been alluded to (Benson et al., 2014; Benson & Scholey, 2014). Their reported t value of 1.77 with 11 degrees of freedom is associated with a p value of .052 (one tailed) or .14 (two tailed), so strictly speaking was not significant. It is unclear why Linden‐Carmichael and Lau‐Barraco (2017) would choose to cite the results of one of two arms in a single study over a well‐conducted meta‐analysis (or indeed several papers by Marczinski's group, which did not find a masking effect). It does, however, illustrate the pitfalls of choosing a single study (or in this case, one dose from a single study) over a more thorough synthesis of the data. The nature of meta‐analyses enables more representative findings to emerge and to indicate which individual studies are outliers. Unfortunately, this is not simply an academic exercise. Expectancies about the effects of drugs can affect their outcomes. In an elegant experiment, Fillmore, Roach, and Rice (2002) showed that when subjects were led to expect that caffeine reversed the effects of alcohol, the consequence was increased impairment when alcohol was mixed with caffeine. Thus, falsely suggesting that masking occurs, or that AMED harms exist if they do not, could have serious real‐world consequences if they are publicized and believed by AMED users. There is a dearth of prospective cohort studies comparing AMED and AO effects, and many of the studies in this field have methodological flaws. Thus, Linden‐Carmichael et al. (this issue) rightly indicate that their recent diary paper has certain advantages. For example, their approach allowed drinking patterns and functional consequences to be captured the next day rather than weeks or months later. This does not mean, however, that the paper is the last word on AMED‐related harms. There are also a number of issues regarding data analysis and reporting. Linden‐Carmichael and Lau‐Barraco (2017) selected N = 122 AMED consumers, described as “heavy drinking, college student CAB users” (note that CAB = caffeinated alcoholic beverage, so includes AMED as well as alcohol with other mixers, specifically “Diet and regular soda.”) Each completed an average of 12.42 entries, allowing, during the 2‐week period, 1,515 opportunities to consume alcohol for the whole sample, of which 389 (25.67%) were taken. The majority (74.04%) of these drinking occasions involved non‐CAB rather than CAB drinks (presented as 288 vs. 101 in Table 1 of the paper). Most of the 101 CAB occasions involved non‐AMED beverages (71 vs. 40 in Table 1). Although, given that 71 + 40 = 111, there may have been 10 occasions when CAB drinker consumed both. Confusingly, the paper states that of “CAB days, cola‐caffeinated alcoholic beverages were consumed on 57.43% of CAB days and alcohol mixed with energy drinks [AMED] were consumed on 39.6% of CAB days.” While the figure for AMED is consistent with Table 1, those for cola‐caffeinated alcoholic beverages are not. Specifically, 57.43% of 101 is 58, whereas Table 1 reports that soda and alcohol drinks were consumed on 71 out of 101 (70.3%) of CAB occasions. The reason for this anomaly is unclear; it may be that the authors have differentiated noncaffeinated soda from caffeinated mixers although elsewhere the paper states that both diet and regular soda were considered “cola‐caffeinated mixers” (p. 884). Forty AMED occasions were recorded over 14 days in 122 AMED consumers. In other words, the majority of this cohort of “heavy drinking, college student CAB users” did not consume CAB or AMED over the 14‐day study period. Taking number of diary completions into account, they consumed AMED on 2.64% of days available to them (extrapolating these data would translate to nine or 10 AMED occasions in a year). Unfortunately, the paper does not specify how many individuals contributed to the 40 AMED occasions, which could range from four individuals each consuming AMED on 10 occasions to 40 individuals each consuming AMED once. We did request this information from the authors, but, at the time of writing this response no data were received. Linden‐Carmichael and Lau‐Barraco (2017) concluded that there were more alcohol‐related harms following AMED than AO. Linden‐Carmichael (this issue) suggest that their study is superior due to their within‐subjects analysis, stating that the “study compared days in which individuals consumed CABs as opposed to days in which they consumed other types of alcohol.” Strictly speaking this is not accurate. It would be more correct to state that the study compared CAB with non‐CAB occasions within a cohort of CAB users. A more appropriate approach would be to conduct a true within‐subjects comparison among only those subjects who experienced both AMED and AO occasions within the 14‐day period. This may be possible because the paper states that there were 50.80% (N = 62) individuals who consumed both non‐CAB and CAB (although the number of AMED users within this subset is not specified). Comparing functional consequences of AMED within the same drinkers (a true within‐subject comparison) over the period of study would have provided useful information regarding the role of AMED in alcohol harms. Findings from the Linden‐Carmichael and Lau‐Barraco (2017) study need to be replicated (perhaps addressing some of the shortcomings outlined above). They do not change the validity of our meta‐analysis and its outcome that “mixing alcohol with energy drink does not affect subjective intoxication and seems unlikely to increase total alcohol consumption, associated risk‐taking behavior, nor other negative alcohol related consequence.” Taken together, we are pleased to have this opportunity to respond to Linden‐Carmichael et al. (this issue) and believe that, as concluded in Verster et al. (2018), “Further research may be necessary to fully reveal the effects of AMED.” CONFLICT OF INTEREST The authors have declared no conflict of interest. DISCLOSURE OF INTERESTS Chris Alford has undertaken sponsored research or provided consultancy, for a number of companies and organizations including Airbus Group Industries, Astra, British Aerospace/BAeSystems, Civil Aviation Authority, Duphar, FarmItalia Carlo Erba, Ford Motor Company, ICI, Innovate UK, Janssen, LERS Synthélabo, Lilly, Lorex/Searle, Ministry of Defense, Quest International, Red Bull GmbH, Rhone‐Poulenc Rorer, and Sanofi Aventis. Sarah Benson has received funding from Red Bull GmbH, Kemin Foods, Sanofi Aventis, and GlaxoSmithKline. Samuel Benrejeb Godefroy's research activities are funded by the Ministry of Agriculture, Fisheries, and Food, Government of Quebec, Canada; the Ministry of Science, Technology, and Innovation, Government of Quebec; Canada's Innovation Foundation; the U.S. Department of Agriculture Foreign Agriculture Service; r‐Biopharm GmbH; and r‐Biopharm Canada Inc. Samuel Godefroy acts as an expert advisor for members of the food and beverage industry, international organizations (the Food and Agriculture Organization of the United Nations, the United Nations Industrial Development Organization, and the World Bank), and international food regulators such as the China National Centre for Food Safety Risk Assessment and consumer organizations such as Food Allergy Canada. Sean Johnson has undertaken sponsored research for Pfizer, AstraZeneca, Merck, Gilead, Novartis, Roche, Red Bull GmbH, the Department for Transport, and Road Safety Trust. Andrew Scholey has received research funding from Abbott Nutriton, Arla Foods, Bayer Healthcare, Cognis, Cyvex, GlaxoSmithKline, Kemin Foods, Naturex, Nestlé, Martek, Masterfoods, Red Bull GmbH, Sanofi, Vesrdure Sciences, and Wrigley and has acted as a consultant/expert advisor to Abbott Nutrition, Barilla, Bayer Healthcare, Danone, Flordis, GlaxoSmithKline Healthcare, Masterfoods, Martek, Neurobrands, and Wrigley. Joris Verster has received grants/research support from the Dutch Ministry of Infrastructure and the Environment, Janssen Research and Development, Nutricia, Takeda, Red Bull, Sequential and has acted as a consultant for Canadian Beverage Association, Centraal Bureau Drogisterijbedrijven, Clinilabs, Coleman Frost, Danone, Deenox, Eisai, Janssen, Jazz, Purdue, Red Bull, Sanofi‐Aventis, Sen‐Jam Pharmaceutical, Sepracor, Takeda, Transcept, Trimbos Institute, Vital Beverages, and ZBiotics. Red Bull GmbH was not involved in the preparation of the manuscript.