A new methodology emulating fact-checker criteria assesses news outlet factuality and bias using LLMs. The approach uses prompts based on fact-checking criteria to elicit and aggregate LLM responses for predictions. Experiments demonstrate improvements over baselines, with error analysis on media popularity and region, and a released dataset/code at https://github.com/mbzuai-nlp/llm-media-profiling.
A study by MBZUAI's Preslav Nakov and Cornell co-authors examines how to develop systems that detect fake news in a landscape where text is generated by humans and machines. The research, presented at the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics, analyzes fake news detectors' ability to identify human- and machine-written content. The study highlights biases in current detectors, which tend to classify machine-written news as fake and human-written news as true. Why it matters: Addressing these biases is crucial as machine-generated content becomes more prevalent in both real and fake news, requiring more nuanced detection methods.
MBZUAI Professor Preslav Nakov believes AI can outpace human fact-checkers in detecting fake news by analyzing language and sentence structure. AI systems can identify common sources of fake news and flag domains for blocking. Nakov's research focuses on disinformation, fact checking, and media bias detection. Why it matters: AI-driven solutions for combating fake news could help mitigate the spread of misinformation and its impact on society, especially in the Arabic-speaking world.