Debiasing Large Language Models toward Social Factors in Online Behavior Analytics through Prompt Knowledge Tuning
arXiv:2603.27057v1 Announce Type: cross Abstract: Attribution theory explains how individuals interpret and attribute others' behavior in a social context by employing personal (dispositional) and impersonal (situational) causality. Large Language Models (LLMs), trained on human-generated corpora, may implicitly mimic this social attribution process in social contexts. However, the extent to which LLMs utilize these causal attributions in their reasoning remains underexplored. Although using reasoning paradigms, such as Chain-of-Thought (CoT), has shown promising results in various tasks, igno — Hossein Salemi, Jitin Krishnan, Hemant Purohit
View PDF HTML (experimental)
Abstract:Attribution theory explains how individuals interpret and attribute others' behavior in a social context by employing personal (dispositional) and impersonal (situational) causality. Large Language Models (LLMs), trained on human-generated corpora, may implicitly mimic this social attribution process in social contexts. However, the extent to which LLMs utilize these causal attributions in their reasoning remains underexplored. Although using reasoning paradigms, such as Chain-of-Thought (CoT), has shown promising results in various tasks, ignoring social attribution in reasoning could lead to biased responses by LLMs in social contexts. In this study, we investigate the impact of incorporating a user's goal as knowledge to infer dispositional causality and message context to infer situational causality on LLM performance. To this end, we introduce a scalable method to mitigate such biases by enriching the instruction prompts for LLMs with two prompt aids using social-attribution knowledge, based on the context and goal of a social media message. This method improves the model performance while reducing the social-attribution bias of the LLM in the reasoning on zero-shot classification tasks for behavior analytics applications. We empirically show the benefits of our method across two tasks-intent detection and theme detection on social media in the disaster domain-when considering the variability of disaster types and multiple languages of social media. Our experiments highlight the biases of three open-source LLMs: Llama3, Mistral, and Gemma, toward social attribution, and show the effectiveness of our mitigation strategies.
Comments: This is a preprint of the accepted paper for publication in IEEE Transactions on Computational Social Systems
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2603.27057 [cs.CL]
(or arXiv:2603.27057v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.27057
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Hossein Salemi [view email] [v1] Sat, 28 Mar 2026 00:15:40 UTC (2,080 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
researchpaperarxiv
AI is Driving Cognitive Surrender Whilst Influencing Confidence Levels
AI has rapidly transformed how people access information and make decisions. Tools like ChatGPT offer speed, convenience and support for everyday tasks, however growing evidence suggested overreliance on AI may influence how we think, reason and evaluate information. The research from the University of Pennsylvania’s Wharton School of Business has reviewed 1,300 subjects use of [ ] The post AI is Driving Cognitive Surrender Whilst Influencing Confidence Levels appeared first on DIGIT .
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Research Papers
Explainability is a must for older adults to trust AI, study shows
Voice-activated, conversational artificial intelligence (AI) agents must provide clear explanations for their suggestions, or older adults aren t likely to trust them. That s one of the main findings from a study by AI Caring on what older adults expect from explainable AI (XAI).
SocioEval: A Template-Based Framework for Evaluating Socioeconomic Status Bias in Foundation Models
As Large Language Models (LLMs) increasingly power decision-making systems across critical domains, understanding and mitigating their biases becomes essential for responsible AI deployment. Although bias assessment frameworks have proliferated for attributes such as race and gender, socioeconomic status bias remains significantly underexplored despite its widespread implications in the real world. We introduce SocioEval, a template-based framework for systematically evaluating socioeconomic bias in foundation models through decision-making tasks. Our hierarchical framework encompasses 8 theme — Divyanshu Kumar, Ishita Gupta, Nitin Aravind Birur
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!