Abstract
Fact-checking remains one of the most prominent ways to counter misinformation. Increasingly so, however, fact-checkers are challenged to match the speed of (mis)information production and increasingly focus on false information at the expense of true information. Could Large Language Models (LLMs) like ChatGPT help enhance the efficiency and expediency of fact-checking processes? We conducted a systematic analysis to measure ChatGPT’s fact-checking performance by submitting 12,784 fact-checked statements to ChatGPT as a zero-shot classification. We found that ChatGPT accurately categorized statements in 72% of cases, with significantly greater accuracy at identifying true claims (80%) than false claims (67%). Overall, ChatGPT did better on statements that have been more recently fact-checked (even after its training date). These findings demonstrate the potential of ChatGPT to help label misinformation. However, LLMs like ChatGPT should not replace the crucial work of human fact-checking experts in upholding the accuracy of information dissemination.
Preprint DOI
License
CC-By Attribution 4.0 International
