Stance Detection in Prediction Markets: Addressing Imbalanced Trader Commentary via Counterfactual Augmentation and Market Context
预测市场中的立场检测:通过反事实增强和市场上下文解决交易者评论不平衡问题
Thomas Mbrice
AI总结 针对预测市场评论中极端不平衡的立场检测问题,提出结合市场上下文和LLM驱动的反事实增强方法,显著提升了少数类(反对立场)的召回率和F1值。
详情
- Comments
- 14 pages, 9 figures
Polymarket等预测市场将群体信念聚合为实时概率估计,交易者在每个市场下方发布的评论包含价格无法捕捉的丰富方向性立场信号。本文首次将立场检测研究应用于预测市场评论,该领域具有极端简短性、交易者特定用语和严重的类别不平衡(仅8.7%的评论反对市场结果)。我们在4×3消融实验中对RoBERTa-base进行微调:四种输入配置({2类, 3类} × {有/无市场上下文})和三种增强条件(基线、50%合成、100%合成)。通过Anthropic API,利用LLM驱动的Pro→Anti反事实翻转生成合成少数类样本。结果表明:(1)市场上下文是影响最大的单一因素,将3类Anti召回率从0.10提升至0.45;(2)反事实增强有条件地有效,在弱配置中提升Anti F1(0.10→0.24),但在强配置中降低性能(2类上下文宏F1:全剂量下从0.68降至0.50);(3)50%增强是最佳剂量,100%始终损害性能。基于注意力的可解释性分析为所有三个发现提供了机制支持。
Prediction markets such as Polymarket aggregate crowd beliefs into real-time probability estimates, and the comments traders post beneath each market contain rich directional stance signals that prices alone cannot capture. This work introduces the first stance detection study applied to prediction market commentary, a domain characterized by extreme brevity, trader- specific vernacular, and severe class imbalance (only 8.7% of comments oppose the market outcome). RoBERTa-base is fine-tuned across a 4 x 3 ablation: four input configurations ({2- class, 3-class} x {with/without market context}) and three augmentation conditions (baseline, 50% synthetic, 100% synthetic). Synthetic minority-class samples are generated via LLM-driven Pro -> Anti counterfactual flips using the Anthropic API. Results show that (1) market context is the single most impactful factor, raising 3-class Anti recall from 0.10 to 0.45; (2) counterfactual augmentation is conditionally effective, improving Anti F1 in weak configurations (0.10 -> 0.24) while degrading strong ones (2-class-ctx macro F1: 0.68 -> 0.50 at full dose); and (3) 50% augmentation is the optimal dose, with 100% consistently hurting performance. Attention-based interpretability analysis provides mechanistic support for all three findings.