Off-Policy Learning with Limited Supply
arXiv:2603.18702v2 Announce Type: replace Abstract: We study off-policy learning (OPL) in contextual bandits, which plays a key role in a wide range of real-world applications such as recommendation systems and online advertising. Typical OPL in contextual bandits assumes an unconstrained environment where a policy can select the same item infinitely. However, in many practical applications, including coupon allocation and e-commerce, limited supply constrains items through budget limits on distributed coupons or inventory restrictions on products. In these settings, greedily selecting the ite — Koichi Tanaka, Ren Kishimoto, Bushun Kawagishi, Yusuke Narita, Yasuo Yamamoto, Nobuyuki Shimizu, Yuta Saito
View PDF HTML (experimental)
Abstract:We study off-policy learning (OPL) in contextual bandits, which plays a key role in a wide range of real-world applications such as recommendation systems and online advertising. Typical OPL in contextual bandits assumes an unconstrained environment where a policy can select the same item infinitely. However, in many practical applications, including coupon allocation and e-commerce, limited supply constrains items through budget limits on distributed coupons or inventory restrictions on products. In these settings, greedily selecting the item with the highest expected reward for the current user may lead to early depletion of that item, making it unavailable for future users who could potentially generate higher expected rewards. As a result, OPL methods that are optimal in unconstrained settings may become suboptimal in limited supply settings. To address the issue, we provide a theoretical analysis showing that conventional greedy OPL approaches may fail to maximize the policy performance, and demonstrate that policies with superior performance must exist in limited supply settings. Based on this insight, we introduce a novel method called Off-Policy learning with Limited Supply (OPLS). Rather than simply selecting the item with the highest expected reward, OPLS focuses on items with relatively higher expected rewards compared to the other users, enabling more efficient allocation of items with limited supply. Our empirical results on both synthetic and real-world datasets show that OPLS outperforms existing OPL methods in contextual bandit problems with limited supply.
Comments: Published as a conference paper at WWW 2026
Subjects:
Machine Learning (cs.LG)
Cite as: arXiv:2603.18702 [cs.LG]
(or arXiv:2603.18702v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.18702
arXiv-issued DOI via DataCite
Submission history
From: Koichi Tanaka [view email] [v1] Thu, 19 Mar 2026 10:01:39 UTC (1,809 KB) [v2] Mon, 30 Mar 2026 02:04:51 UTC (1,813 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
researchpaperarxiv
Major 4-day workweek study suggests that when we work 5 days we spend one doing basically nothing
Research says workers can get as much done in a 33-hour week as in 38 hours. Essentially, those of us on a five-day week are filling up our days with time-wasting activities.
UKRI Deems Turing Institute Not Yet Satisfactory
UK Research and Innovation (UKRI) found that the Alan Turing Institute s strategic alignment and value for money are not yet satisfactory in a review of the AI research body s performance. The Turing Institute has dealt with a tumultuous year, with its head stepping down amid pushback from staff complaining about a toxic work environment. The [ ] The post UKRI Deems Turing Institute Not Yet Satisfactory appeared first on DIGIT .
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!