Search

Results for "social meaning"

SocialMaze: A Benchmark for Evaluating Social Reasoning in Large Language Models

arXiv · May 29

MBZUAI researchers introduce SocialMaze, a new benchmark for evaluating social reasoning capabilities in large language models (LLMs). SocialMaze includes six diverse tasks across social reasoning games, daily-life interactions, and digital community platforms, emphasizing deep reasoning, dynamic interaction, and information uncertainty. Experiments show that LLMs vary in handling dynamic interactions, degrade under uncertainty, but can be improved via fine-tuning on curated reasoning examples.

A Benchmark Study of Contrastive Learning for Arabic Social Meaning

arXiv · Oct 22

This paper presents a benchmark study of contrastive learning (CL) methods applied to Arabic social meaning tasks like sentiment analysis and dialect identification. The study compares state-of-the-art supervised CL techniques against vanilla fine-tuning across a range of tasks. Results indicate that CL methods outperform vanilla fine-tuning in most cases and demonstrate data efficiency. Why it matters: This work highlights the potential of contrastive learning for improving performance in Arabic NLP, especially in low-resource scenarios.

ADAB: Arabic Dataset for Automated Politeness Benchmarking -- A Large-Scale Resource for Computational Sociopragmatics

arXiv · Feb 14

The paper introduces ADAB (Arabic Politeness Dataset), a new annotated Arabic dataset for politeness detection collected from online platforms. The dataset covers Modern Standard Arabic and multiple dialects (Gulf, Egyptian, Levantine, and Maghrebi). It contains 10,000 samples across 16 politeness categories and achieves substantial inter-annotator agreement (kappa = 0.703). Why it matters: This dataset addresses the under-explored area of Arabic-language resources for politeness detection, which is crucial for culturally-aware NLP systems.