Using ChatGPT in Statistical Analysis : Recommendations for JACC Journals

Abstract

The paper published in JACC: Advances pointed out the awareness of potential pitfalls and the critical evaluation associated with using large language models such as generative pretrained transforms (ChatGPT). 1 In July 2023, Conroy raised the question of whether using ChatGPT to generate an entire article from scratch might be of any use. 2 However, the recommendations published to date are missing an important aspect, which is the correctness of the advice obtained in terms of broad statistical analysis. These days, despite the publication of editorial advice, there is unfortunately a low level of statistical analysis carried out. This may be due to the problem of finding statistical reviewers, the unfamiliarity with the assumptions needed to apply various statistical tests, and little practical teaching of biostatistics. One of the most recent published study results indicates that only 40% of accepted manuscripts on various aspects of COVID-19 meet statistical validity. 3 An article published in JACC: Basic to Translational Science indicated that ChatGPT should be able to generate more detailed responses. 4 Unfortunately, as it turns out, using ChatGPT to obtain answers to statistical analysis questions can lead to the interpretation of inappropriate test results. I suggest the following considerations when using ChatGPT for statistics: 1. When obtaining advice from the ChatGPT website related to conducting statistical analysis, it is essential to pay attention to the assumptions necessary to apply specific statistical tests. 2. In situations where there is divergent information on statistical analysis, the selection of appropriate tests should be supported by appropriate references. 3. At a time when there is less statistical analysis being carried out and, at the same time, an increased frequency of use of language models, there should be an even greater emphasis on the use of statistical reviewers/editors.