AI Agent

Technical Report: Evaluating Goal Drift in Language Model Agents

Rauno Arike, Elizabeth Donoway, Henning Bartsch, Marius Hobbhahn

专题命中 Agent评测 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Multimodal Situational Safety

Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Anderson Compalas, Dawn Song, Xin Eric Wang

机构 * University of California, Santa Cruz（加州大学圣克ruz分校） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.CL

Comments ICLR 2025 Camera Ready

AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks

Charlotte Siska, Anush Sankaran

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.CL

PaperBench: Evaluating AI's Ability to Replicate AI Research

Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, Johannes Heidecke, Amelia Glaese, Tejal Patwardhan

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI、cs.CL

Comments 30 pages, 14 figures

VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM

Yuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng, Boqiang Zhang, Long Li, Xin Li, Deli Zhao, Wenqiao Zhang, Yueting Zhuang, Jianke Zhu, Lidong Bing

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 14 figures, technical report

Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents

Rongwu Xu, Xiaojian Li, Shuo Chen, Wei Xu

专题命中 Agent评测 ：agent(abstract)；agentic(abstract)；分类 cs.AI、cs.CL

Comments Please visit https://llm-catastrophic-risks.github.io for a quick tour of our research. Our code is available at https://github.com/pillowsofwind/LLM-CBRN-Risks

Retrieval Augmented Generation for Topic Modeling in Organizational Research: An Introduction with Empirical Demonstration

Gerion Spielberger, Florian M. Artinger, Jochen Reb, Rudolf Kerschreiter

专题命中 Agent评测 ：agent(abstract)；agentic(abstract)；分类 cs.AI、cs.LG

Comments 30 pages, 4 figures

Rapfi: Distilling Efficient Neural Network for the Game of Gomoku

Zhanggen Jin, Haobin Duan, Zhiyang Hang

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI、cs.LG

Video Action Differencing

James Burgess, Xiaohan Wang, Yuhui Zhang, Anita Rau, Alejandro Lozano, Lisa Dunlap, Trevor Darrell, Serena Yeung-Levy

专题命中 Agent评测 ：workflow(abstract)；agentic(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2025 (International Conference on Learning Representations) Project page: http://jmhb0.github.io/viddiff Benchmark: https://huggingface.co/datasets/jmhb/VidDiffBench

Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning

Shangding Gu, Laixi Shi, Muning Wen, Ming Jin, Eric Mazumdar, Yuejie Chi, Adam Wierman, Costas Spanos

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions

Yuxuan Liu, Hongda Sun, Wei Liu, Jian Luan, Bo Du, Rui Yan

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.CL

Comments Accepted by KDD2025 Research Track

Text2World: Benchmarking Large Language Models for Symbolic World Model Generation

Mengkang Hu, Tianxing Chen, Yude Zou, Yuheng Lei, Qiguang Chen, Ming Li, Yao Mu, Hongyuan Zhang, Wenqi Shao, Ping Luo

专题命中 Agent评测 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.CL

Comments Project page: https://text-to-world.github.io/

Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator

Frederic Kirstein, Terry Ruas, Bela Gipp

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.CL

Journal ref COLING 2025 Industry Track

Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations

Oualid Bougzime, Samir Jabbar, Christophe Cruz, Frédéric Demoly

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Comments 54 pages, 7 figures

CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments

Kung-Hsiang Huang, Akshara Prabhakar, Sidharth Dhawan, Yixin Mao, Huan Wang, Silvio Savarese, Caiming Xiong, Philippe Laban, Chien-Sheng Wu

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI、cs.CL

Comments NAACL 2025

SkyRover: A Modular Simulator for Cross-Domain Pathfinding

Wenhui Ma, Wenhao Li, Bo Jin, Changhong Lu, Xiangfeng Wang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Comments 9 pages

MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents

Yixing Jiang, Kameron C. Black, Gloria Geng, Danny Park, James Zou, Andrew Y. Ng, Jonathan H. Chen

专题命中 Agent评测 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Illuminating Spaces: Deep Reinforcement Learning and Laser-Wall Partitioning for Architectural Layout Generation

Reza Kakooee, Benjamin Dillenburger

专题命中 Agent评测 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Dissecting Adversarial Robustness of Multimodal LM Agents

Chen Henry Wu, Rishi Shah, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, Aditi Raghunathan

专题命中 Agent评测 ：agent(abstract)；autonomous agent(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2025. Also oral at NeurIPS 2024 Open-World Agents Workshop

Simulation Streams: A Programming Paradigm for Controlling Large Language Models and Building Complex Systems with Generative AI

Peter Sunehag, Joel Z. Leibo

专题命中 Agent评测 ：agent(abstract)；agentic(abstract)；分类 cs.AI、cs.SE

Comments Technical report accompanying the release of code on GitHub

LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models

Hang Yang, Hao Chen, Hui Guo, Yineng Chen, Ching-Sheng Lin, Shu Hu, Jinrong Hu, Xi Wu, Xin Wang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.CL

Intelligent System for Automated Molecular Patent Infringement Assessment

Yaorui Shi, Sihang Li, Taiyan Zhang, Xi Fang, Jiankun Wang, Zhiyuan Liu, Guojiang Zhao, Zhengdan Zhu, Zhifeng Gao, Renxin Zhong, Linfeng Zhang, Guolin Ke, Weinan E, Hengxing Cai, Xiang Wang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

MDD-5k: A New Diagnostic Conversation Dataset for Mental Disorders Synthesized via Neuro-Symbolic LLM Agents

Congchi Yin, Feng Li, Shu Zhang, Zike Wang, Jun Shao, Piji Li, Jianhua Chen, Xun Jiang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.CL

Comments Accepted by the 39th Annual AAAI Conference on Artificial Intelligence

Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization

Yanhu Wang, Muhammad Muzammil Afzal, Zhengyang Li, Jie Zhou, Chenyuan Feng, Shuaishuai Guo, Tony Q. S. Quek

专题命中 Agent评测 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.CL

Choice Between Partial Trajectories: Disentangling Goals from Beliefs

Henrik Marklund, Benjamin Van Roy

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI、cs.LG

AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents

Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, Honglak Lee

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.CL、cs.LG

Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks

Yun Qu, Boyuan Wang, Jianzhun Shao, Yuhang Jiang, Chen Chen, Zhenbin Ye, Lin Liu, Junfeng Yang, Lin Lai, Hongyang Qin, Minwen Deng, Juchao Zhuo, Deheng Ye, Qiang Fu, Wei Yang, Guang Yang, Lanxiao Huang, Xiangyang Ji

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Learning to Cooperate with Humans using Generative Agents

Yancheng Liang, Daphne Chen, Abhishek Gupta, Simon S. Du, Natasha Jaques

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation

Claude Formanek, Callum Rhys Tilbury, Louise Beyers, Jonathan Shock, Arnu Pretorius

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Comments Accepted at 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks

Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models

Chakshu Moar, Faraz Tahmasebi, Michael Pellauer, Hyoukjun Kwon

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.CL、cs.LG