视觉大模型 / VLM

Knowledge-Based Counterfactual Queries for Visual Question Answering

Theodoti Stoikou, Maria Lymperaiou, Giorgos Stamou

专题命中 视觉问答 ：visual question answering(title,abstract)

Journal ref AAAI MAKE 2023

Learning the meanings of function words from grounded language using a visual question answering model

Eva Portelance, Michael C. Frank, Dan Jurafsky

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Published in Cognitive Science 2024

EVJVQA Challenge: Multilingual Visual Question Answering

Ngan Luu-Thuy Nguyen, Nghia Hieu Nguyen, Duong T. D Vo, Khanh Quoc Tran, Kiet Van Nguyen

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments VLSP2022 EVJVQA challenge

Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering

Pascal Tilli, Ngoc Thang Vu

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Accepted at LREC-COLING 2024

Robotic Applications of Pre-Trained Vision-Language Models to Various Recognition Behaviors

Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

专题命中 视觉问答 ：vision-language model(title,abstract)

Comments Accepted at Humanoids2023

Cross-modal Retrieval for Knowledge-based Visual Question Answering

Paul Lerner, Olivier Ferret, Camille Guinaudeau

专题命中 视觉问答 ：visual question answering(title,abstract)

Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond

Zhecan Wang, Long Chen, Haoxuan You, Keyang Xu, Yicheng He, Wenhao Li, Noel Codella, Kai-Wei Chang, Shih-Fu Chang

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2023

Journal ref EMNLP 2023

OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamese

Nghia Hieu Nguyen, Duong T. D. Vo, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments submitted to Elsevier

Delving Deeper into Cross-lingual Visual Question Answering

Chen Liu, Jonas Pfeiffer, Anna Korhonen, Ivan Vulić, Iryna Gurevych

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Findings of EACL 2023

Modular Visual Question Answering via Code Generation

Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang, Arsha Nagrani, Cordelia Schmid, Andy Zeng, Trevor Darrell, Dan Klein

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments ACL 2023

HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Language

Shantipriya Parida, Idris Abdulmumin, Shamsuddeen Hassan Muhammad, Aneesh Bose, Guneet Singh Kohli, Ibrahim Said Ahmad, Ketan Kotwal, Sayan Deb Sarkar, Ondřej Bojar, Habeebah Adamu Kakudi

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Accepted at ACL 2023 as a long paper (Findings)

Document Collection Visual Question Answering

Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny

专题命中 视觉问答 ：visual question answering(title,abstract)

Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering

Chenxi Whitehouse, Tillman Weyde, Pranava Madhyastha

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Findings of EACL 2023

Curriculum Script Distillation for Multilingual Visual Question Answering

Khyathi Raghavi Chandu, Alborz Geramifard

专题命中 视觉问答 ：visual question answering(title,abstract)

Retrieval Augmented Visual Question Answering with Outside Knowledge

Weizhe Lin, Bill Byrne

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Accepted to appear at the main conference of EMNLP 2022

Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision

Khalil Mrini, Harpreet Singh, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole

专题命中 视觉问答 ：grounding(title,abstract)

Comments Accepted as Main Conference Long paper at COLING 2022

Equivariant and Invariant Grounding for Video Question Answering

Yicong Li, Xiang Wang, Junbin Xiao, Tat-Seng Chua

专题命中 视觉问答 ：grounding(title,abstract)

Comments MM22

xGQA: Cross-Lingual Visual Question Answering

Jonas Pfeiffer, Gregor Geigle, Aishwarya Kamath, Jan-Martin O. Steitz, Stefan Roth, Ivan Vulić, Iryna Gurevych

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Findings of ACL 2022

Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillation

Humair Raj Khan, Deepak Gupta, Asif Ekbal

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Accepted in EMNLP-Findings (2021)

Passage Retrieval for Outside-Knowledge Visual Question Answering

Chen Qu, Hamed Zamani, Liu Yang, W. Bruce Croft, Erik Learned-Miller

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments Accepted to SIGIR'21 as a short paper

Cross-Modal Self-Attention with Multi-Task Pre-Training for Medical Visual Question Answering

Haifan Gong, Guanqi Chen, Sishuo Liu, Yizhou Yu, Guanbin Li

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments ICMR '21: ACM International Conference on Multimedia Retrieval, Taipei, Taiwan, August 21-24, 2021

Improving Visual Question Answering by Referring to Generated Paragraph Captions

Hyounghun Kim, Mohit Bansal

专题命中 视觉问答 ：visual question answering(title,abstract)

Comments ACL 2019 (7 pages)

Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering

Claudio Greco, Barbara Plank, Raquel Fernández, Raffaella Bernardi

专题命中 视觉问答 ：visual question answering(title,abstract)

Leveraging Medical Visual Question Answering with Supporting Facts

Tomasz Kornuta, Deepta Rajan, Chaitanya Shivade, Alexis Asseman, Ahmet S. Ozcan

专题命中 视觉问答 ：visual question answering(title)；分类 cs.CV、cs.AI、cs.LG

Comments Working notes from the ImageCLEF 2019 VQA-Med competition

What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering

重排序器所见：面向长文档多模态问答的多方面页面标注

Guanchen Wu, Jiayuan Ding, Subhabrata Mukherjee, Carl Yang

机构 * Emory University（埃默里大学） ； Hippocratic AI（希波克拉底人工智能公司） 纠错

专题命中 视觉问答 ：VLM(abstract,abstract_cn)；visual question answering(abstract)；分类 cs.AI

AI总结 本文针对长文档多模态问答的重排序瓶颈，提出含Trident-R与Trident-S组件的Trident模型，通过多方面页面标注提升检索与生成性能，在多数据集上取得显著效果。

VideoVIBE: A Video-Grounded Diagnostic Benchmark for One-Shot Interactive Website Generation

VideoVIBE：用于一次性交互式网站生成的基于视频的诊断基准

Jiajun Xu, Yanghao Zhou, Jingyun Liao, Yu Bai, Jinxing Zhou, Chengliang Liu, Changsen Yuan, Bo Wang, Qian Liu

专题命中 视觉问答 ：MLLM(summary_cn,abstract_cn)；分类 cs.CV

AI总结 针对现有一次性交互式网站生成质量评估的不足，提出VideoVIBE基准与V2Lens多智能体系统，实验显示V2Lens可提升Video MLLM的评估性能。

ChronoVision: Temporal Reasoning via Latent State Reconstruction

ChronoVision：基于潜在状态重构的时序推理

Yifan Shen, Jian Xu, Boyi Li, Yuner Zhang, Tianjiao Yu, Bingxuan Li, Houze Yang, Rushi Wang, Xu Cao

专题命中 视觉问答 ：grounding(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV

AI总结 针对多模态大语言模型时序推理不足的问题，本文提出ChronoVision框架，引入Vbvr-VQA数据集，实验显示其在相关基准上取得最优性能。

DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning

DocTrace：面向可追溯的长文档视觉问答的分层证据图推理方法

Le Xiang, Zhicheng Guan, Hong Chen, Xiaocong Lin, Zhenghua Lei, Teng Hu, Bolei He, Long Zeng

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)；MLLM(abstract_cn)；分类 cs.AI

AI总结 本文提出DocTrace分层框架，将长文档视觉问答建模为显式证据图推理问题，经两阶段训练优化后，在三个基准上优于现有模型，且推理可追溯。

OncoTriad-QA: A Patient-Level Radiology-Pathology-Genomics Benchmark for Pan-Cancer Reasoning

OncoTriad-QA：用于泛癌推理的患者级放射学-病理学-基因组学基准

Ahnaf Munir, Dannong Wang, Michael W. McDonald, Mubarak Shah, Pegah Khosravi, Yu Tian

专题命中 视觉问答 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.AI

AI总结 本文提出OncoTriad-QA多模态癌症问答基准及OncoVLM模型，实验显示OncoVLM经微调后在泛癌问答任务上优于现有模型，该基准可用于相关模型的训练与评估。

Location-Aware Fine-Grained Representation Learning for Medical Vision Foundation Models

面向医学视觉基础模型的位置感知细粒度表示学习

Myeongkyun Kang, Yanting Yang, Xiaoxiao Li

机构 * The University of British Columbia（不列颠哥伦比亚大学） ； Vector Institute（矢量研究所） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)；grounding(abstract)；分类 cs.CV

AI总结 本研究提出基于位置感知细粒度表示学习的医学视觉基础模型LoFi，构建大规模医学定位数据集MedG，在多项医学视觉任务中性能优于现有模型。