MBZUAI Professor Preslav Nakov has developed FRAPPE, an interactive website that analyzes news articles to identify persuasion techniques. FRAPPE helps users understand framing, persuasion, and propaganda at an aggregate level, across different news outlets and countries. Presented at EACL, FRAPPE uses 23 specific techniques categorized into six broader buckets, such as 'attack on reputation' and 'manipulative wording'. Why it matters: The tool addresses the increasing difficulty in discerning factual information from disinformation, providing a means to identify biases in news media from different countries.
A new methodology emulating fact-checker criteria assesses news outlet factuality and bias using LLMs. The approach uses prompts based on fact-checking criteria to elicit and aggregate LLM responses for predictions. Experiments demonstrate improvements over baselines, with error analysis on media popularity and region, and a released dataset/code at https://github.com/mbzuai-nlp/llm-media-profiling.
This paper introduces a new task: detecting propaganda techniques in code-switched text. The authors created and released a corpus of 1,030 English-Roman Urdu code-switched texts annotated with 20 propaganda techniques. Experiments show the importance of directly modeling multilinguality and using the right fine-tuning strategy for this task.
MBZUAI student Zain Muhammad Mujahid is researching methods to detect media bias using NLP and LLMs. His approach profiles bias across media outlets using LLMs like ChatGPT to predict bias based on 16 identifiers. The research aims to develop a tool that instantly provides a bias profile for a given media URL. Why it matters: This research has the potential to combat misinformation and enhance media literacy in the region by providing tools to identify biased reporting, and it is expanding to Arabic and other languages.
MBZUAI researchers presented new resources at EMNLP for improving the factuality of LLMs, including a web application for fact-checking LLM-generated text and benchmarks for evaluating automated fact-checkers. They found that current automated fact-checkers miss nearly 40% of false claims generated by LLMs. The study breaks down the fact-checking process into eight tasks, including decomposition and decontextualization, to identify where systems fail. Why it matters: This work addresses a critical challenge in the deployment of LLMs by providing tools and methods for improving their reliability and trustworthiness, which is essential for widespread adoption in sensitive applications.