Financial misinformation detection

Project description

From the original project page. Results and time references describe that version of the work.

Overview

In this paper, we present an instruction-tuned GPT-4o-mini model for detecting financial misinformation and providing reasoned explanations for the classifications. Our approach enhances GPT-4o-mini's ability to identify false, true, and inadequately supported financial claims by leveraging a carefully designed instruction-tuning pipeline and specialized prompting strategy. The model processes financial claims along with their contextual information to perform dual classification and explanation generation tasks.

Working with the FIN-FACT dataset, which contains diverse financial sector claims spanning areas like income, finance, economy, budget, taxes, and debt, we demonstrate that our instruction-tuned model achieves strong performance while requiring minimal training data. By instruction-tuning on only 918 balanced samples, our model achieves state-of-the-art results with a micro F1 score of 0.788 and a ROUGE-1 score of 0.743 on the private test set, securing fourth place in the Financial Misinformation Detection (FMD) shared task.

Our approach outperforms several baseline models, including zero-shot applications of large language models like ChatGPT, LLaMA variants, and other instruction-tuned models. The framework's strength lies in its ability to not only classify financial claims accurately but also generate high-quality explanations justifying its classifications, as evidenced by strong ROUGE scores across different metrics. This makes our solution particularly valuable for real-world applications where transparency and reasoning are crucial alongside accurate detection.