AI Agent

MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs

Xuannan Liu, Zekun Li, Peipei Li, Huaibo Huang, Shuhan Xia, Xing Cui, Linzhi Huang, Weihong Deng, Zhaofeng He

专题命中 Agent评测 ：agent(abstract)；tool-use(abstract)；分类 cs.CL

Comments Accepted by ICLR 2025, Project page: https://liuxuannan.github.io/MMFakeBench.github.io/

MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Lilian Weng, Aleksander Mądry

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.CL

Comments 10 pages, 17 pages appendix. Equal contribution by first seven authors, authors randomized. ICLR version

Graphy'our Data: Towards End-to-End Modeling, Exploring and Generating Report from Raw Data

Longbin Lai, Changwei Luo, Yunkai Lou, Mingchen Ju, Zhengyi Yang

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI

Comments 4 pages

AgentFL: Scaling LLM-based Fault Localization to Project-Level Context

Yihao Qin, Shangwen Wang, Yiling Lou, Jinhao Dong, Kaixin Wang, Xiaoling Li, Xiaoguang Mao

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.SE

Comments Added a comment to refer to the published version

UAV-assisted Internet of Vehicles: A Framework Empowered by Reinforcement Learning and Blockchain

Ahmed Alagha, Maha Kadadha, Rabeb Mizouni, Shakti Singh, Jamal Bentahar, Hadi Otrok

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Auto-Evaluation: A Critical Measure in Driving Improvements in Quality and Safety of AI-Generated Lesson Resources

Hannah-Beth Clark, Margaux Dowland, Laura Benton, Reka Budai, Ibrahim Kaan Keskin, Emma Searle, Matthew Gregory, Mark Hodierne, William Gayne, John Roberts

专题命中 Agent评测 ：agent(abstract)；planning(abstract)；分类 cs.AI

Comments 27 pages, Part of MIT Open Learning AI and Open Education Initiative Series, published Jan 2025 https://aiopeneducation.pubpub.org/pub/i36sncz8/release/3?readingCollection=06969c6d

VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks

Lawrence Jang, Yinheng Li, Dan Zhao, Charles Ding, Justin Lin, Paul Pu Liang, Rogerio Bonatti, Kazuhito Koishida

专题命中 Agent评测 ：agent(abstract)；agentic(abstract)；分类 cs.AI

MDCrow: Automating Molecular Dynamics Workflows with Large Language Models

Quintina Campbell, Sam Cox, Jorge Medina, Brittany Watterson, Andrew D. White

专题命中 Agent评测 ：agent(abstract)；agentic(abstract)；分类 cs.AI

Generative AI-Enhanced Cooperative MEC of UAVs and Ground Stations for Unmanned Surface Vehicles

Jiahao You, Ziye Jia, Chao Dong, Qihui Wu, Zhu Han

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Do as We Do, Not as You Think: the Conformity of Large Language Models

Zhiyuan Weng, Guikun Chen, Wenguan Wang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.CL

Comments ICLR 2025 (Oral). Code: https://github.com/Zhiyuan-Weng/BenchForm

Invisible Traces: Using Hybrid Fingerprinting to identify underlying LLMs in GenAI Apps

Devansh Bhardwaj, Naman Mishra

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.LG

Game Theory with Simulation in the Presence of Unpredictable Randomisation

Vojtech Kovarik, Nathaniel Sauerberg, Lewis Hammond, Vincent Conitzer

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI

Survey and Improvement Strategies for Gene Prioritization with Large Language Models

Matthew Neeley, Guantong Qi, Guanchu Wang, Ruixiang Tang, Dongxue Mao, Chaozhong Liu, Sasidhar Pasupuleti, Bo Yuan, Fan Xia, Pengfei Liu, Zhandong Liu, Xia Hu

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Comments 11 pages, 4 figures, 10 pages of supplementary figures

NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notes

Junda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou, Rumeng Li, Xun Wang, Yucheng Xu, Hong Yu

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.CL

Journal ref Findings of the Association for Computational Linguistics: ACL 2024

Evaluating GenAI for Simplifying Texts for Education: Improving Accuracy and Consistency for Enhanced Readability

Stephanie L. Day, Jacapo Cirica, Steven R. Clapp, Veronika Penkova, Amy E. Giroux, Abbey Banta, Catherine Bordeau, Poojitha Mutteneni, Ben D. Sawyer

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.CL

Comments 64 pages, 9 tables, 6 figures, and supplemental materials

Measuring Diversity of Game Scenarios

Yuchen Li, Ziqi Wang, Qingquan Zhang, Bo Yuan, Jialin Liu

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Will you donate money to a chatbot? The effect of chatbot anthropomorphic features and persuasion strategies on willingness to donate

Ekaterina Novozhilova, Jiacheng Huang, Le He, Ziling Li, James Cummings

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI

Comments 13 pages, 2 figures

SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMAC

Yue Deng, Yan Yu, Weiyu Ma, Zirui Wang, Wenhui Zhu, Jian Zhao, Yin Zhang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

PerSphere: A Comprehensive Framework for Multi-Faceted Perspective Retrieval and Summarization

Yun Luo, Yingjie Li, Xiangkun Hu, Qinglin Qi, Fang Guo, Qipeng Guo, Zheng Zhang, Yue Zhang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.CL

Cultural Evolution of Cooperation among LLM Agents

Aron Vallinder, Edward Hughes

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI

Comments 15 pages, 6 figures

Towards Foundation-model-based Multiagent System to Accelerate AI for Social Impact

Yunfan Zhao, Niclas Boehmer, Aparna Taneja, Milind Tambe

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons

Andrew Szot, Bogdan Mazoure, Omar Attia, Aleksei Timofeev, Harsh Agrawal, Devon Hjelm, Zhe Gan, Zsolt Kira, Alexander Toshev

专题命中 Agent评测 ：agent(abstract)；planning(abstract)；分类 cs.LG

CityX: Controllable Procedural Content Generation for Unbounded 3D Cities

Shougao Zhang, Mengqi Zhou, Yuxi Wang, Chuanchen Luo, Rongyu Wang, Yiwei Li, Zhaoxiang Zhang, Junran Peng

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Hacking CTFs with Plain Agents

Rustem Turtayev, Artem Petrov, Dmitrii Volkov, Denis Volk

专题命中 Agent评测 ：agent(abstract)；tool use(abstract)；分类 cs.AI

Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale

Tianyue Ou, Frank F. Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, Shuyan Zhou

专题命中 Agent评测 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI

AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, Florian Tramèr

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.LG

Comments Updated version after fixing a bug in the Llama implementation and updating the travel suite

AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Test

U. León-Domínguez, E. D. Flores-Flores, A. J. García-Jasso, M. K. Gómez-Cuellar, D. Torres-Sánchez, A. Basora-Marimon

专题命中 Agent评测 ：agent(abstract)；AI agent(abstract)；分类 cs.AI

Comments 25 pages, 2 figures, 7 tables

Simulate and Optimise: A two-layer mortgage simulator for designing novel mortgage assistance products

Leo Ardon, Benjamin Patrick Evans, Deepeka Garg, Annapoorani Lakshmi Narayanan, Makada Henry-Nickie, Sumitra Ganesh

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Comments Accepted at the 5th ACM International Conference on AI in Finance

FairStream: Fair Multimedia Streaming Benchmark for Reinforcement Learning Agents

Jannis Weil, Jonas Ringsdorf, Julian Barthel, Yi-Ping Phoebe Chen, Tobias Meuser

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI

Benchmarking Large Language Models for Image Classification of Marine Mammals

Yijiashun Qi, Shuzhang Cai, Zunduo Zhao, Jiaming Li, Yanbin Lin, Zhiqiang Wang

专题命中 Agent评测 ：agent(abstract)；multi-agent(abstract)；分类 cs.CL

Comments ICKG 2024