Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents
反事实评估揭示临床LLM和智能体的隐藏能力画像
机构 * Protege Data Lab(Protege数据实验室)
专题命中 医学数据与评测 :clinical LLM(title);clinical AI(abstract);分类 cs.LG
AI总结 提出因果敏感性评分(CSS),通过沿五个临床维度变异肿瘤病例来评估模型是否按预期方向更新推荐,发现与覆盖度指标排名相反,并揭示所有前沿模型在手术状态干预上的安全盲点。
Comments Accepted to RLEval @ ACM CAIS 2026 (Workshop on Methods and RL Environments for Evaluating AI Agents) and selected for an invited talk based on reviewer ratings. 4-page short paper + appendix