Cross-Domain Fake News Classification via RoBERTa Fine-Tuning on Hybrid LIAR-FakeNewsNet Corpus with Deployable Web Inference

Abstract

The rapid spread of misinformation on social media platforms escalates a serious threat to world socio-political stability, health and financial security. While current automated fake news detection models boast impressive accuracy scores, these are often due to memorisation of the training domain rather than genuine generalisation to unseen examples. This study tackles the pressing need for domain-robust misinformation and disinformation detection through the creation of a highly adaptive and deployable classification system. The challenge stems from the severe drop in performance when exposed to the domain shift, especially with the emergence of generative language models that can perform style-based attacks. The proposed solution employs a RoBERTa-base transformer model, fine-tuned on a large, multi-domain corpus of 37,143 instances. This dataset mixes the short, politically focused statements of the LIAR dataset with the long, multi-topic articles from the FakeNewsNet dataset and ensures cross-domain generalisation. Using full-parameter fine-tuning and evaluation across different domains, the model attained 86.7% accuracy, 87.3% macro precision and 86.4% macro F1-score on a diverse test corpus, surpassing traditional baselines. Importantly, the assessment unveiled a conservative labelling tendency with a 90.4% precision in detecting fake news and a 93.1% recall in detecting genuine news which prevents the severe consequences of mistakenly accusing legitimate news. Moreover, the design breaks out of research-only models with its integration into a real-time web service (based on Flask), providing sub-second inferences with confidence scores. This approach simultaneously addresses the problem of cross-domain vulnerability of state-of-the-art natural language processing models and demands deployable interpretability, setting a new benchmark for the implementation of natural language processing models in the fight against advanced, online misinformation.