Social IntelligenceAI that understands, generates, and takes part in human interaction

Adding video to an MLLM gives surprisingly inconsistent gains on social tasks. We locate the attention heads responsible for cross-modal grounding and inject a social-aware attention bias derived from speaker locations, with no trainable parameters, in LLaVA-NeXT-Video, Qwen2.5-VL, and InternVL3.

The weak link is alignment. In multi-speaker scenes, a speaker’s visual tokens and their words barely attend to each other; restoring that alignment sets state of the art on four social tasks.

MIDA asks whether each statement in a game of One Night Ultimate Werewolf is true, false, or neutral, with ground truth reconstructed from manually annotated night actions. We benchmark 12 MLLMs and introduce Social Chain-of-Thought reasoning with a Dynamic Social Epistemic Memory.

Even GPT-4o cannot reliably separate truth from deception. Models struggle to track what each player knows, and adding structured epistemic memory consistently helps.

Grounded Personality Reasoning requires a model to support each Big Five rating with observable behaviour. MM-OCEAN provides 1,104 videos and 5,320 cue-grounding questions, and a three-tier evaluation of 27 MLLMs.

The Prejudice Gap. 51% of correct ratings are not grounded in retrieved behavioural cues, so models often reach the right score for the wrong reason.

Mock politeness is polite in form but hostile in function. ZhenhuanBench turns 47 hours of expert commentary on Empresses in the Palace into 1,592 questions on speaker intent, emotion–language mismatch, and how each listener takes a remark. CPR-GRPO, a Compress–Probe–Replenish loop, reuses the same annotations to train an 8B model.

Reading the listeners is the hardest part. The best of 34 LLMs scores 73.48% overall but 58.37% on how listeners react; with only 338 gold traces, CPR-GRPO brings an 8B model to 67.0 on the multiple-choice tasks.