多模态大模型

MetaDesigner: Advancing Artistic Typography Through AI-Driven, User-Centric, and Multilingual WordArt Synthesis

Jun-Yan He, Zhi-Qi Cheng, Chenyang Li, Jingdong Sun, Qi He, Wangmeng Xiang, Hanyuan Chen, Jin-Peng Lan, Xianhui Lin, Kang Zhu, Bin Luo, Yifeng Geng, Xuansong Xie, Alexander G. Hauptmann

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.AI、cs.MM

Comments Accepted by ICLR 2025, Project: https://modelscope.cn/studios/WordArt/WordArt

Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens

Ziwei Shan, Yaoyu He, Chengfeng Zhao, Jiashen Du, Jingyan Zhang, Qixuan Zhang, Jingyi Yu, Lan Xu

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments First three authors contribute equally. Project page: https://koyui.github.io/mojito/

The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning

Sheila Schoepp, Masoud Jafaripour, Yingyue Cao, Tianpei Yang, Fatemeh Abdollahi, Shadan Golestan, Zahin Sufiyan, Osmar R. Zaiane, Matthew E. Taylor

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 4 figures

A Roadmap for Embodied and Social Grounding in LLMs

Sara Incao, Carlo Mazzola, Giulia Belgiovine, Alessandra Sciutti

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted Version of a conference paper presented at Robophilosophy Conference 2024

Journal ref Incao, S., Mazzola, C., Belgiovine, G., Sciutti, A., 2025, A Roadmap for Embodied and Social Grounding in LLMs. In J. Seibt, P. Fazekas, & O. S. Quick (Eds.), Social Robots with AI: Prospects, Risks, and Responsible Methods, IOS Press

Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining

Zhiqi Ge, Juncheng Li, Xinglei Pang, Minghe Gao, Kaihang Pan, Wang Lin, Hao Fei, Wenqiao Zhang, Siliang Tang, Yueting Zhuang

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Developing Enhanced Conversational Agents for Social Virtual Worlds

D. Griol, A. Sanchis, J. M. Molina, Z. Callejas

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Neurocomputing 2019

Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks

Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, Heng Ji

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.CL

CrossVIT-augmented Geospatial-Intelligence Visualization System for Tracking Economic Development Dynamics

Yanbing Bai, Jinhua Su, Bin Qiao, Xiaoran Ma

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Mediating Modes of Thought: LLM's for design scripting

Moritz Rietschel, Fang Guo, Kyle Steinfeld

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Published at ACADIA 2024

BIP3D: Bridging 2D Images and 3D Perception for Embodied Intelligence

Xuewu Lin, Tianwei Lin, Lichao Huang, Hongyu Xie, Zhizhong Su

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Neurosymbolic Graph Enrichment for Grounded World Models

Stefano De Giorgis, Aldo Gangemi, Alessandro Russo

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

A Review of Large Language Models and Autonomous Agents in Chemistry

Mayk Caldas Ramos, Christopher J. Collison, Andrew D. White

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CL、cs.AI

To Ask or Not to Ask? Detecting Absence of Information in Vision and Language Navigation

Savitha Sam Abraham, Sourav Garg, Feras Dayoub

专题命中 多模态Agent ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at WACV 2025

Unlocking the Archives: Using Large Language Models to Transcribe Handwritten Historical Documents

Mark Humphries, Lianne C. Leddy, Quinn Downton, Meredith Legace, John McConnell, Isabella Murray, Elizabeth Spence

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 29 Pages, 11 Tables, 2 Figures

MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making

Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, Hae Won Park

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CL、cs.AI

Reliable, Routable, and Reproducible: Collection of Pedestrian Pathways at Statewide Scale

Yuxiang Zhang, Bill Howe, Anat Caspi

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:2303.02323

Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding

Yue Fan, Lei Ding, Ching-Chen Kuo, Shan Jiang, Yang Zhao, Xinze Guan, Jie Yang, Yi Zhang, Xin Eric Wang

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.CL

OSCAR: Operating System Control via State-Aware Reasoning and Re-Planning

Xiaoqiang Wang, Bang Liu

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

Details Make a Difference: Object State-Sensitive Neurorobotic Task Planning

Xiaowen Sun, Xufeng Zhao, Jae Hee Lee, Wenhao Lu, Matthias Kerzel, Stefan Wermter

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments ICANN24, Switzerland

Instruction-Guided Visual Masking

Jinliang Zheng, Jianxiong Li, Sijie Cheng, Yinan Zheng, Jiaming Li, Jihao Liu, Yu Liu, Jingjing Liu, Xianyuan Zhan

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024

TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation

Junli Ren, Yikai Liu, Yingru Dai, Junfeng Long, Guijin Wang

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Published on CoRL 2024

Language Agents as Optimizable Graphs

Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, Jürgen Schmidhuber

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Project Website: https://gptswarm.org ; Github Repo: https://github.com/metauto-ai/gptswarm . In Forty-first International Conference on Machine Learning (2024)

Learning to Model the World with Language

Jessy Lin, Yuqing Du, Olivia Watkins, Danijar Hafner, Pieter Abbeel, Dan Klein, Anca Dragan

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments ICML 2024. Website: https://dynalang.github.io/

UMass-BioNLP at MEDIQA-M3G 2024: DermPrompt -- A Systematic Exploration of Prompt Engineering with GPT-4V for Dermatological Diagnosis

Parth Vashisht, Abhilasha Lodha, Mukta Maddipatla, Zonghai Yao, Avijit Mitra, Zhichao Yang, Junda Wang, Sunjae Kwon, Hong Yu

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted at NAACL-ClinicalNLP workshop 2024

Evaluating Tool-Augmented Agents in Remote Sensing Platforms

Simranjit Singh, Michael Fore, Dimitrios Stamoulis

专题命中 多模态Agent ：image-text(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2024 Machine Learning for Remote Sensing (ML4RS) Workshop

The Essential Role of Causality in Foundation World Models for Embodied AI

Tarun Gupta, Wenbo Gong, Chao Ma, Nick Pawlowski, Agrin Hilmkil, Meyer Scetbon, Marc Rigter, Ade Famoti, Ashley Juan Llorens, Jianfeng Gao, Stefan Bauer, Danica Kragic, Bernhard Schölkopf, Cheng Zhang

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CL、cs.AI

GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilots

Simranjit Singh, Michael Fore, Dimitrios Stamoulis

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Earthvision 2024, CVPR Workshop

Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model

Zhonghan Zhao, Ke Ma, Wenhao Chai, Xuan Wang, Kewei Chen, Dongxu Guo, Yanting Zhang, Hongwei Wang, Gaoang Wang

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments arXiv admin note: text overlap with arXiv:2403.08282

Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning

Jaewoo Jeong, Daehee Park, Kuk-Jin Yoon

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 2024 CVPR Highlight

Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs

Yusuke Mikami, Andrew Melnik, Jun Miura, Ville Hautamäki

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 2 figures