AI Agent

InSTA: Towards Internet-Scale Training For Agents

Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu, Ruslan Salakhutdinov

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Amazon（亚马逊） 纠错

专题命中 GUI与网页智能体 ：agent(abstract)；agentic(abstract)；分类 cs.AI、cs.LG

Comments Improved results, zero-shot transfer to Web Voyager

InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners

Yuhang Liu, Pengxiang Li, Congkai Xie, Xavier Hu, Xiaotian Han, Shengyu Zhang, Hongxia Yang, Fei Wu

机构 * Zhejiang University（浙江大学） ； Dalian University of Technology（大连理工大学） ； Reallm Labs（Reallm实验室） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.CL

Comments 10 pages, 3 figures, work in progress

Breaking the Data Barrier -- Building GUI Agents Through Task Generalization

Junlei Zhang, Zichen Ding, Chang Ma, Zijie Chen, Qiushi Sun, Zhenzhong Lan, Junxian He

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.CL

Comments 24 pages, 11 figures

TRISHUL: Towards Region Identification and Screen Hierarchy Understanding for Large VLM based GUI Agents

Kunal Singh, Shreyas Singh, Mukund Khanna

专题命中 GUI与网页智能体 ：agent(abstract)；agentic(abstract)；分类 cs.AI、cs.LG

Comments 8 pages 5 figures

Language Model Powered Digital Biology with BRAD

Joshua Pickard, Ram Prakash, Marc Andrew Choi, Natalie Oliven, Cooper Stansbury, Jillian Cwycyshyn, Alex Gorodetsky, Alvaro Velasquez, Indika Rajapakse

专题命中 GUI与网页智能体 ：agent(abstract)；agentic(abstract)；分类 cs.AI、cs.SE

Comments 12 pages, 3 figures, 1 table. See: https://github.com/Jpickard1/BRAD

On the Effects of Data Scale on UI Control Agents

Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, Oriana Riva

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024 (Datasets and Benchmarks)

OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization

Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Hongming Zhang, Tianqing Fang, Zhenzhong Lan, Dong Yu

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.CL

Multimodal Auto Validation For Self-Refinement in Web Agents

Ruhana Azam, Tamer Abuelsaad, Aditya Vempaty, Ashish Jagmohan

专题命中 GUI与网页智能体 ：agent(abstract)；workflow(abstract)；分类 cs.AI、cs.SE

DPN: Decoupling Partition and Navigation for Neural Solvers of Min-max Vehicle Routing Problems

Zhi Zheng, Shunyu Yao, Zhenkun Wang, Xialiang Tong, Mingxuan Yuan, Ke Tang

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Almanac Copilot: Towards Autonomous Electronic Health Record Navigation

Cyril Zakka, Joseph Cho, Gracia Fahed, Rohan Shad, Michael Moor, Robyn Fong, Dhamanpreet Kaur, Vishnu Ravi, Oliver Aalami, Roxana Daneshjou, Akshay Chaudhari, William Hiesinger

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Mobile Sequencers

Cem Bozsahin

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.CL

Embodied Agents for Efficient Exploration and Smart Scene Description

Roberto Bigazzi, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.CL

Comments Accepted by IEEE International Conference on Robotics and Automation (ICRA 2023)

Multimodal Web Navigation with Instruction-Finetuned Foundation Models

Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Comments Accepted to ICLR 2024. Website: https://sites.google.com/view/mm-webnav/

Autonomous Navigation in Complex Environments

Andrew Gerstenslager, Jomol Lewis, Liam McKenna, Poorva Patel

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Comments 7 pages, 3 figures, independent paper

Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation

Jiachen Li, David Isele, Kanghoon Lee, Jinkyoo Park, Kikuo Fujimura, Mykel J. Kochenderfer

专题命中 GUI与网页智能体 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 14 figures

Verified Compositional Neuro-Symbolic Control for Stochastic Systems with Temporal Logic Tasks

Jun Wang, Haojun Chen, Zihe Sun, Yiannis Kantaros

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2209.06130

Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions

Yuxing Long, Xiaoqi Li, Wenzhe Cai, Hao Dong

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.CL

Comments Submitted to ICRA 2024

Federated Few-Shot Learning for Mobile NLP

Dongqi Cai, Shangguang Wang, Yaozong Wu, Felix Xiaozhu Lin, Mengwei Xu

专题命中 GUI与网页智能体 ：planning(abstract)；workflow(abstract)；分类 cs.CL、cs.LG

Comments MobiCom 2023

BEVBert: Multimodal Map Pre-training for Language-guided Navigation

Dong An, Yuankai Qi, Yangguang Li, Yan Huang, Liang Wang, Tieniu Tan, Jing Shao

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.CL

Comments ICCV 2023, project page: https://github.com/MarSaKi/VLN-BEVBert

Towards real-world navigation with deep differentiable planners

Shu Ishida, João F. Henriques

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Comments Published in CVPR 2022 (Conference on Computer Vision and Pattern Recognition)

Reinforced Imitative Graph Learning for Mobile User Profiling

Dongjie Wang, Pengyang Wang, Yanjie Fu, Kunpeng Liu, Hui Xiong, Charles E. Hughes

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Comments TKDE Under Review

MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation

Saim Wani, Shivansh Patel, Unnat Jain, Angel X. Chang, Manolis Savva

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://shivanshpatel35.github.io/multi-ON/ ; the first three authors contributed equally

Relational Graph Learning for Crowd Navigation

Changan Chen, Sha Hu, Payam Nikdel, Greg Mori, Manolis Savva

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Comments Accepted to IROS 2020. Added links to codes and video demo

BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps

Wang Zhu, Hexiang Hu, Jiacheng Chen, Zhiwei Deng, Vihan Jain, Eugene Ie, Fei Sha

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.CL

Comments Accepted by ACL 2020

On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based Approach

Nicolò Botteghi, Beril Sirmacek, Khaled A. A. Mustafa, Mannes Poel, Stefano Stramigioli

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Learning Navigation Subroutines from Egocentric Videos

Ashish Kumar, Saurabh Gupta, Jitendra Malik

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Mastering emergent language: learning to guide in simulated navigation

Mathijs Mul, Diane Bouchacourt, Elia Bruni

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.CL

Sim-to-Real Optimization of Complex Real World Mobile Network with Imperfect Information via Deep Reinforcement Learning from Self-play

Yongxi Tan, Jin Yang, Xin Chen, Qitao Song, Yunjun Chen, Zhangxiang Ye, Zhenqiang Su

专题命中 GUI与网页智能体 ：agent(abstract)；multi-agent(abstract)；分类 cs.AI、cs.LG

Comments Accepted by NIPS 2018 Workshop

Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning

Oleksii Zhelo, Jingwei Zhang, Lei Tai, Ming Liu, Wolfram Burgard

专题命中 GUI与网页智能体 ：agent(abstract)；autonomous agent(abstract)；分类 cs.AI、cs.LG

Comments 5 pages, 4 figures; to be published in MLPC 2018

Semi-parametric Topological Memory for Navigation

Nikolay Savinov, Alexey Dosovitskiy, Vladlen Koltun

专题命中 GUI与网页智能体 ：agent(abstract)；planning(abstract)；分类 cs.AI、cs.LG

Comments Published at International Conference on Learning Representations (ICLR) 2018. Project website at https://sites.google.com/view/SPTM