多模态大模型

SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entities

Hyunjong Ok, Taeho Kil, Sukmin Seo, Jaeho Lee

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 7 figures, NAACL 2024

MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements

Lisong C. Sun, Neel P. Bhatt, Jonathan C. Liu, Zhiwen Fan, Zhangyang Wang, Todd E. Humphreys, Ufuk Topcu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Project Webpage: https://vita-group.github.io/MM3DGS-SLAM

Neural Multimodal Topic Modeling: A Comprehensive Evaluation

Felipe González-Pizarro, Giuseppe Carenini

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Camera-Ready for LREC-COLING 2024 (Long Paper)

MOGAM: A Multimodal Object-oriented Graph Attention Model for Depression Detection

Junyeop Cha, Seoyun Kim, Dongjae Kim, Eunil Park

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 3 figures, 4 tables

A Unified Model for Longitudinal Multi-Modal Multi-View Prediction with Missingness

Boqi Chen, Junier Oliva, Marc Niethammer

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Aria-NeRF: Multimodal Egocentric View Synthesis

Jiankai Sun, Jianing Qiu, Chuanyang Zheng, John Tucker, Javier Yu, Mac Schwager

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models

Rocktim Jyoti Das, Simeon Emilov Hristov, Haonan Li, Dimitar Iliyanov Dimitrov, Ivan Koychev, Preslav Nakov

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document

Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, Xiang Bai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoning

Deepanway Ghosal, Vernon Toh Yan Han, Chia Yew Ken, Soujanya Poria

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Pink: Unveiling the Power of Referential Comprehension for Multi-modal LLMs

Shiyu Xuan, Qingpei Guo, Ming Yang, Shiliang Zhang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

MMAPS: End-to-End Multi-Grained Multi-Modal Attribute-Aware Product Summarization

Tao Chen, Ze Lin, Hui Li, Jiayi Ji, Yiyi Zhou, Guanbin Li, Rongrong Ji

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.MM

Comments LREC-COLING 2024.11 pages, 4 figures

PALO: A Polyglot Large Multimodal Model for 5B People

Muhammad Maaz, Hanoona Rasheed, Abdelrahman Shaker, Salman Khan, Hisham Cholakal, Rao M. Anwer, Tim Baldwin, Michael Felsberg, Fahad S. Khan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Technical Report of PALO

GroundingGPT:Language Enhanced Multi-modal Grounding Model

Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, Yiqing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Van Tu Vu, Zhida Huang, Tao Wang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Modality-Aware and Shift Mixer for Multi-modal Brain Tumor Segmentation

Zhongzhen Huang, Linda Wei, Shaoting Zhang, Xiaofan Zhang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

REWIND Dataset: Privacy-preserving Speaking Status Segmentation from Multimodal Body Movement Signals in the Wild

Jose Vargas Quiros, Chirag Raman, Stephanie Tan, Ekin Gedik, Laura Cabrera-Quiros, Hayley Hung

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Exploring Multimodal Large Language Models for Radiology Report Error-checking

Jinge Wu, Yunsoo Kim, Eva C. Keller, Jamie Chow, Adam P. Levine, Nikolas Pontikos, Zina Ibrahim, Paul Taylor, Michelle C. Williams, Honghan Wu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, Jiaqi Li, Xiaoze Liu, Jeff Z. Pan, Ningyu Zhang, Huajun Chen

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Ongoing work; 41 pages (Main Text), 55 pages (Total), 11 Tables, 13 Figures, 619 citations; Paper list is available at https://github.com/zjukg/KG-MM-Survey

MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing

Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, Bozhong Tian

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages

Text Role Classification in Scientific Charts Using Multimodal Transformers

Hye Jin Kim, Nicolas Lell, Ansgar Scherp

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Exploring Perceptual Limitation of Multimodal Large Language Models

Jiarui Zhang, Jinyi Hu, Mahyar Khayatkhoei, Filip Ilievski, Maosong Sun

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 14 figures, 3 tables

TransGPT: Multi-modal Generative Pre-trained Transformer for Transportation

Peng Wang, Xiang Wei, Fangxu Hu, Wenjuan Han

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

On the Out-Of-Distribution Generalization of Multimodal Large Language Models

Xingxuan Zhang, Jiansheng Li, Wenjing Chu, Junjia Hai, Renzhe Xu, Yuqing Yang, Shikai Guan, Jiazheng Xu, Peng Cui

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

2AFC Prompting of Large Multimodal Models for Image Quality Assessment

Hanwei Zhu, Xiangjie Sui, Baoliang Chen, Xuelin Liu, Peilin Chen, Yuming Fang, Shiqi Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysis

Yingshu Li, Yunyi Liu, Zhanyu Wang, Xinyu Liang, Lei Wang, Lingqiao Liu, Leyang Cui, Zhaopeng Tu, Longyue Wang, Luping Zhou

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

CBVS: A Large-Scale Chinese Image-Text Benchmark for Real-World Short Video Search Scenarios

Xiangshuo Qiao, Xianxin Li, Xiaozhe Qu, Jie Zhang, Yang Liu, Yu Luo, Cihang Jin, Jin Ma

专题命中 多模态评测 ：image-text(title,abstract)；分类 cs.CV、cs.MM

KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning

Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh, Godawari Sudhakar Rao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments AAAI 2024

Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning

Mustafa Shukor, Alexandre Rame, Corentin Dancette, Matthieu Cord

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments ICLR 2024. Project Page: https://evalign-icl.github.io/

Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading

Dunyuan Xu, Xi Wang, Jinyue Cai, Pheng-Ann Heng

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perception

Yipo Huang, Quan Yuan, Xiangfei Sheng, Zhichao Yang, Haoning Wu, Pengfei Chen, Yuzhe Yang, Leida Li, Weisi Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Multimodal Crowd Counting with Pix2Pix GANs

Muhammad Asif Khan, Hamid Menouar, Ridha Hamila

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted version of the paper in 19th International Conference on Computer Vision Theory and Applications (VISAPP), Rome, Italy, 27-29 Feb, 2024,