多模态大模型

Bottom-up Anytime Discovery of Generalised Multimodal Graph Patterns for Knowledge Graphs

Xander Wilcke, Rick Mourits, Auke Rijpma, Richard Zijdeman

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

MACeIP: A Multimodal Ambient Context-enriched Intelligence Platform in Smart Cities

Truong Thanh Hung Nguyen, Phuc Truong Loc Nguyen, Monica Wachowicz, Hung Cao

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

Comments 4 pages, 6 figures, IEEE/IEIE ICCE-Asia 2024

Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale

Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, Lawrence Jang, Zack Hui

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.AI

TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering

Yiqing Shen, Zan Chen, Michail Mamalakis, Yungeng Liu, Tianbin Li, Yanzhou Su, Junjun He, Pietro Liò, Yu Guang Wang

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.AI

Target-Oriented Object Grasping via Multimodal Human Guidance

Pengwei Xie, Siang Chen, Dingchang Hu, Yixiang Dai, Kaiqin Yang, Guijin Wang

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024 Workshop on Assistive Computer Vision and Robotics (ACVR 2024)

CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registration

Gongxin Yao, Yixin Xuan, Xinyang Li, Yu Pan

专题命中 多模态Agent ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024

Conversational AI Multi-Agent Interoperability, Universal Open APIs for Agentic Natural Language Multimodal Communications

Diego Gosmar, Deborah A. Dahl, Emmett Coin

专题命中 多模态Agent ：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Comments 22 pages, 8 figures

MEIA: Multimodal Embodied Perception and Interaction in Unknown Environments

Yang Liu, Xinshuai Song, Kaixuan Jiang, Weixing Chen, Jingzhou Luo, Guanbin Li, Liang Lin

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV

Comments Codes will be available at https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List

Disentangled Acoustic Fields For Multimodal Physical Scene Understanding

Jie Yin, Andrew Luo, Yilun Du, Anoop Cherian, Tim K. Marks, Jonathan Le Roux, Chuang Gan

专题命中 多模态Agent ：multimodal(title,abstract)；分类 eess.AS

AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence

Alireza Ghafarollahi, Markus J. Buehler

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.AI

Activity-based and agent-based Transport model of Melbourne (AToM): an open multi-modal transport simulation model for Greater Melbourne

Afshin Jafari, Dhirendra Singh, Alan Both, Mahsa Abdollahyar, Lucy Gunn, Steve Pemberton, Billie Giles-Corti

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.AI

Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational Agents

Mehdi Arjmand, Farnaz Nouraei, Ian Steenstra, Timothy Bickmore

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL

A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist

Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, Longtao Zheng, Xinrun Wang, Bo An

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

From Perfect to Noisy World Simulation: Customizable Embodied Multi-modal Perturbations for SLAM Robustness Benchmarking

Xiaohao Xu, Tianyi Zhang, Sibo Wang, Xiang Li, Yongqi Chen, Ye Li, Bhiksha Raj, Matthew Johnson-Roberson, Xiaonan Huang

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CV

Comments 50 pages. arXiv admin note: substantial text overlap with arXiv:2402.08125

CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation

Xinbei Ma, Zhuosheng Zhang, Hai Zhao

专题命中 多模态Agent ：MLLM(title)；multimodal(abstract)；分类 cs.CL

Comments ACL'2024 Findings

MCD: A Model-Agnostic Counterfactual Search Method For Multi-modal Design Modifications

Lyle Regenwetter, Yazan Abu Obaideh, Faez Ahmed

专题命中 多模态Agent ：multi-modal(title)；cross-modal(abstract)；分类 cs.AI

Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives

Sheng Luo, Wei Chen, Wanxin Tian, Rui Liu, Luanxuan Hou, Xiubao Zhang, Haifeng Shen, Ruiqi Wu, Shuyi Geng, Yi Zhou, Ling Shao, Yi Yang, Bojun Gao, Qun Li, Guobin Wu

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to IEEE Transactions on Intelligent Vehicles(T-IV). 24 pages, 9 figures, 1 table

Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI

Song Yaoxian, Sun Penglei, Liu Haoyu, Li Zhixu, Song Wei, Xiao Yanghua, Zhou Xiaofang

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

Comments 14 pages, 6 figures

Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld

Yijun Yang, Tianyi Zhou, Kanxue Li, Dapeng Tao, Lusong Li, Li Shen, Xiaodong He, Jing Jiang, Yuhui Shi

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception

Yiran Qin, Enshen Zhou, Qichang Liu, Zhenfei Yin, Lu Sheng, Ruimao Zhang, Yu Qiao, Jing Shao

专题命中 多模态Agent ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments Accepted to CVPR2024

Towards Robust Multi-Modal Reasoning via Model Selection

Xiangyan Liu, Rongxue Li, Wei Ji, Tao Lin

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted by ICLR 2024

Multimodal Web Navigation with Instruction-Finetuned Foundation Models

Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted to ICLR 2024. Website: https://sites.google.com/view/mm-webnav/

Supporting Experts with a Multimodal Machine-Learning-Based Tool for Human Behavior Analysis of Conversational Videos

Riku Arakawa, Kiyosu Maeda, Hiromu Yakura

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV

Dynamic Transformer Architecture for Continual Learning of Multimodal Tasks

Yuliang Cai, Mohammad Rostami

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV

MIMo: A Multi-Modal Infant Model for Studying Cognitive Development

Dominik Mattern, Pierre Schumacher, Francisco M. López, Marcel C. Raabe, Markus R. Ernst, Arthur Aubret, Jochen Triesch

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.AI

Comments 11 pages, 8 figures. Submitted to IEEE Transactions on Congnitive and Developmental Systems (TCDS)

The Importance of Multimodal Emotion Conditioning and Affect Consistency for Embodied Conversational Agents

Che-Jui Chang, Samuel S. Sohn, Sen Zhang, Rajath Jayashankar, Muhammad Usman, Mubbasir Kapadia

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models

Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, Xiaojian Ma, Yitao Liang

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

Comments update project page

Multi-modal Instance Refinement for Cross-domain Action Recognition

Yuan Qing, Naixing Wu, Shaohua Wan, Lixin Duan

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by PRCV 2023

R2H: Building Multimodal Navigation Helpers that Respond to Help Requests

Yue Fan, Jing Gu, Kaizhi Zheng, Xin Eric Wang

专题命中 多模态Agent ：multimodal(title)；multi-modal(abstract)；分类 cs.CL

Comments EMNLP 2023

Project Aria: A New Tool for Egocentric Multi-Modal AI Research

Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattof, Arnie Yuan, Bilal Souti, Brighid Meredith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel DeTone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Duncan Frost, Edward Miller, Elias Mueggler, Evgeniy Oleinik, Fan Zhang, Guruprasad Somasundaram, Gustavo Solaira, Harry Lanaras, Henry Howard-Jenkins, Huixuan Tang, Hyo Jin Kim, Jaime Rivera, Ji Luo, Jing Dong, Julian Straub, Kevin Bailey, Kevin Eckenhoff, Lingni Ma, Luis Pesqueira, Mark Schwesinger, Maurizio Monge, Nan Yang, Nick Charron, Nikhil Raina, Omkar Parkhi, Peter Borschowa, Pierre Moulon, Prince Gupta, Raul Mur-Artal, Robbie Pennington, Sachin Kulkarni, Sagar Miglani, Santosh Gondi, Saransh Solanki, Sean Diener, Shangyi Cheng, Simon Green, Steve Saarinen, Suvam Patra, Tassos Mourikis, Thomas Whelan, Tripti Singh, Vasileios Balntas, Vijay Baiyya, Wilson Dreewes, Xiaqing Pan, Yang Lou, Yipu Zhao, Yusuf Mansour, Yuyang Zou, Zhaoyang Lv, Zijian Wang, Mingfei Yan, Carl Ren, Renzo De Nardi, Richard Newcombe

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CV