Researchers from MBZUAI and Monash University presented a study at EMNLP 2024 examining LLMs' ability to interpret empathy, emotion, and morality in written stories. The study builds on a framework for modeling empathic similarity between narratives, using the EmpathicStories dataset. They are exploring ways to improve LLMs' capabilities with complex concepts like empathy, especially for applications in fields like healthcare. Why it matters: Enhancing LLMs with empathic understanding could lead to more effective and human-centered AI applications, particularly in sensitive domains requiring nuanced communication.
MBZUAI researchers presented new resources at EMNLP for improving the factuality of LLMs, including a web application for fact-checking LLM-generated text and benchmarks for evaluating automated fact-checkers. They found that current automated fact-checkers miss nearly 40% of false claims generated by LLMs. The study breaks down the fact-checking process into eight tasks, including decomposition and decontextualization, to identify where systems fail. Why it matters: This work addresses a critical challenge in the deployment of LLMs by providing tools and methods for improving their reliability and trustworthiness, which is essential for widespread adoption in sensitive applications.
A talk will present two projects related to the use of NLP for estimating a client’s depression severity and well-being. The first project examines emotional coherence between the subjective experience of emotions and emotion expression in therapy using transformer-based emotion recognition models. The second project proposes a semantic pipeline to study depression severity in individuals based on their social media posts by exploring different aggregation methods to answer one of four Beck Depression Inventory (BDI) options per symptom. Why it matters: This research explores how NLP techniques can be applied to mental health assessment, potentially offering new tools for diagnosis and treatment monitoring.
This study investigates the ability of six large language models, including Jais, Mistral, and GPT-4o, to mimic human emotional expression in English and personality markers in Arabic. The researchers evaluated whether machine classifiers could distinguish between human-authored and AI-generated texts and assessed the emotional/personality traits exhibited by the LLMs. Results indicate that AI-generated texts are distinguishable from human-authored ones, with classification performance impacted by paraphrasing, and that LLMs encode affective signals differently than humans. Why it matters: The findings have implications for authorship attribution, affective computing, and the responsible deployment of AI, especially in under-resourced languages like Arabic.
Thamar Solorio of MBZUAI served as general chair of EMNLP 2024, which hosted over 4,000 attendees. MBZUAI researchers presented nearly 50 studies, including one co-authored by Solorio and Monojit Choudhury that received an Outstanding Paper Award. Key themes included cultural awareness, machine-generated content detection, and LLM empathy and cultural representation. Why it matters: MBZUAI's strong presence at EMNLP highlights its growing influence in the international NLP research community and its focus on culturally aware AI.