语言大模型 / LLM - arXivDaily 专题

2408.02657 2025-04-25 cs.CV 82%

Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining

Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yi Xin, Xinyue Li, Qi Qin, Yu Qiao, Hongsheng Li, Peng Gao

专题命中预训练与数据：pretraining(title,abstract)；SFT(abstract)

Comments Code available at: https://github.com/Alpha-VLLM/Lumina-mGPT

2503.15485 2025-04-09 cs.CV cs.AI cs.CL cs.LG 82%

TULIP: Towards Unified Language-Image Pretraining

Zineng Tang, Long Lian, Seun Eisape, XuDong Wang, Roei Herzig, Adam Yala, Alane Suhr, Trevor Darrell, David M. Chan

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments (v2) Clarified fine-tuning process, updated appendix

2405.00740 2025-04-01 cs.CV cs.AI cs.CL cs.LG 82%

Modeling Caption Diversity in Contrastive Vision-Language Pretraining

Samuel Lavoie, Polina Kirichenko, Mark Ibrahim, Mahmoud Assran, Andrew Gordon Wilson, Aaron Courville, Nicolas Ballas

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 8 figures, 7 tables, to be published at ICML2024

2503.19157 2025-03-26 cs.CV 82%

HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Models

Mingzhen Huang, Fu-Jen Chu, Bugra Tekin, Kevin J Liang, Haoyu Ma, Weiyao Wang, Xingyu Chen, Pierre Gleize, Hongfei Xue, Siwei Lyu, Kris Kitani, Matt Feiszli, Hao Tang

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)

2503.18407 2025-03-26 cs.CV 82%

VTD-CLIP: Video-to-Text Discretization via Prompting CLIP

Wencheng Zhu, Yuexin Wang, Hongxuan Li, Pengfei Zhu, Qinghua Hu

专题命中预训练与数据：prompting(title)；language model(abstract)；pretraining(abstract)

2503.15406 2025-03-25 cs.CV 82%

Visual Persona: Foundation Model for Full-Body Human Customization

Jisu Nam, Soowon Son, Zhan Xu, Jing Shi, Difan Liu, Feng Liu, Aashish Misraa, Seungryong Kim, Yang Zhou

专题命中预训练与数据：foundation model(title,abstract)；language model(abstract)

Comments CVPR 2025, Project page is available at https://cvlab-kaist.github.io/Visual-Persona

2503.15141 2025-03-20 cs.CV 82%

Object-Centric Pretraining via Target Encoder Bootstrapping

Nikola Đukić, Tim Lebailly, Tinne Tuytelaars

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)

Comments ICLR 2025

2503.12332 2025-03-18 cs.CV 82%

VideoMAP: Toward Scalable Mamba-based Video Autoregressive Pretraining

Yunze Liu, Peiran Wu, Cheng Liang, Junxiao Shen, Limin Wang, Li Yi

专题命中预训练与数据：pretraining(title)；large language model(abstract)；language model(abstract)

2503.09860 2025-03-14 cs.CV 82%

Foundation X: Integrating Classification, Localization, and Segmentation through Lock-Release Pretraining Strategy for Chest X-ray Analysis

Nahid Ul Islam, DongAo Ma, Jiaxuan Pang, Shivasakthi Senthil Velan, Michael Gotway, Jianming Liang

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)

Comments Accepted by WACV 2025

2409.02060 2025-03-04 cs.CL cs.AI cs.LG 82%

OLMoE: Open Mixture-of-Experts Language Models

Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, Hannaneh Hajishirzi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 63 pages (24 main), 36 figures, 17 tables

2410.01264 2025-03-03 cs.CV 82%

Backdooring Vision-Language Models with Out-Of-Distribution Data

Weimin Lyu, Jiachen Yao, Saumya Gupta, Lu Pang, Tao Sun, Lingjie Yi, Lijie Hu, Haibin Ling, Chao Chen

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)

Comments ICLR 2025

2406.05127 2025-02-27 cs.CV 82%

Towards Semantic Equivalence of Tokenization in Multimodal LLM

Shengqiong Wu, Hao Fei, Xiangtai Li, Jiayi Ji, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)

Comments ICLR-2025. The project page: https://chocowu.github.io/SeTok-web/

2502.11058 2025-02-18 cs.DC 82%

DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization

Zhenheng Tang, Zichen Tang, Junlin Huang, Xinglin Pan, Rudan Yan, Yuxin Wang, Amelie Chi Zhou, Shaohuai Shi, Xiaowen Chu, Bo Li

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)

2501.16065 2025-02-12 cs.CV 82%

CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

Huazhong Zhao, Lei Qi, Xin Geng

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)

Comments Accepted by IEEE TIFS

2502.00435 2025-02-04 cs.CV 82%

SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models

Chuc Man Duc, Hiromichi Fukui

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)

2412.12612 2025-01-27 cs.CL cs.AI cs.IR cs.LG 82%

Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework

Aman Tiwari, Shiva Krishna Reddy Malay, Vikas Yadav, Masoud Hashemi, Sathwik Tejaswi Madhusudhan

专题命中预训练与数据：LLM(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NAACL 2025 main conference

2403.15796 2025-01-16 cs.CL cs.AI cs.LG 82%

Understanding Emergent Abilities of Language Models from the Loss Perspective

Zhengxiao Du, Aohan Zeng, Yuxiao Dong, Jie Tang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 8 figures. Accepted in NeurIPS 2024

2501.07396 2025-01-14 cs.CV 82%

Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models

Yasiru Ranasinghe, Vibashan VS, James Uplinger, Celso De Melo, Vishal M. Patel

专题命中预训练与数据：language model(title,abstract)；prompting(abstract)

2305.14014 2024-12-25 cs.CV 82%

CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model

Shuai Zhao, Ruijie Quan, Linchao Zhu, Yi Yang

专题命中预训练与数据：language model(title,abstract)；foundation model(abstract)

Comments Accepted by T-IP. A PyTorch re-implementation is at https://github.com/VamosC/CLIP4STR (Credit on GitHub@VamosC)

2407.15211 2024-12-17 cs.CL cs.AI cs.CR cs.CV cs.LG 82%

Failures to Find Transferable Image Jailbreaks Between Vision-Language Models

Rylan Schaeffer, Dan Valentine, Luke Bailey, James Chua, Cristóbal Eyzaguirre, Zane Durante, Joe Benton, Brando Miranda, Henry Sleight, John Hughes, Rajashree Agrawal, Mrinank Sharma, Scott Emmons, Sanmi Koyejo, Ethan Perez

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024 Workshops: RBFM (Best Paper), Frontiers in AdvML (Oral), Red Teaming GenAI (Oral), SoLaR (Spotlight), SATA

2409.14485 2024-12-11 cs.CV 82%

Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding

Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, Bo Zhao

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)

2411.16793 2024-11-27 cs.CV q-bio.GN 82%

ST-Align: A Multimodal Foundation Model for Image-Gene Alignment in Spatial Transcriptomics

Yuxiang Lin, Ling Luo, Ying Chen, Xushi Zhang, Zihui Wang, Wenxian Yang, Mengsha Tong, Rongshan Yu

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)

2406.10395 2024-11-08 eess.IV cs.CV q-bio.NC 82%

BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation

Joseph Cox, Peng Liu, Skylar E. Stolte, Yunchao Yang, Kang Liu, Kyle B. See, Huiwen Ju, Ruogu Fang

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)

Comments 19 pages, 5 figures, to be published in Medical Image Analysis

2405.18415 2024-11-05 cs.CV cs.AI cs.CL cs.LG 82%

Why are Visually-Grounded Language Models Bad at Image Classification?

Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, Serena Yeung-Levy

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at NeurIPS 2024

2410.21969 2024-10-30 cs.CV 82%

BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays

Yang Zhou, Tan Li Hui Faith, Yanyu Xu, Sicong Leng, Xinxing Xu, Yong Liu, Rick Siow Mong Goh

专题命中预训练与数据：pretraining(title,abstract)；prompting(abstract)

Comments Accepted to NeurIPS24 Datasets and Benchmarks Track

2410.16257 2024-10-22 cs.CV 82%

Elucidating the design space of language models for image generation

Xuantong Liu, Shaozhe Hao, Xianbiao Qi, Tianyang Hu, Jun Wang, Rong Xiao, Yuan Yao

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)

Comments Project page: https://pepper-lll.github.io/LMforImageGeneration/

2410.14509 2024-10-21 cs.CV 82%

CLIP-VAD: Exploiting Vision-Language Models for Voice Activity Detection

Andrea Appiani, Cigdem Beyan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)

2410.09141 2024-10-15 cs.CL cs.AI cs.IR cs.LG 82%

ACER: Automatic Language Model Context Extension via Retrieval

Luyu Gao, Yunyi Zhang, Jamie Callan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2406.01020 2024-10-08 cs.CV 82%

ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining

Daekyu Kwon, Dongyoung Kim, Sehwan Ki, Younghyun Jo, Hyong-Euk Lee, Seon Joo Kim

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)

2311.16201 2024-09-26 cs.CV cs.AI cs.CL cs.LG 82%

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation

Yuhui Zhang, Brandon McKinzie, Zhe Gan, Vaishaal Shankar, Alexander Toshev

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at EMNLP 2024 Main Conference

1. 预训练与数据 12379 篇

Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining

TULIP: Towards Unified Language-Image Pretraining

Modeling Caption Diversity in Contrastive Vision-Language Pretraining

HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Models

VTD-CLIP: Video-to-Text Discretization via Prompting CLIP

Visual Persona: Foundation Model for Full-Body Human Customization

Object-Centric Pretraining via Target Encoder Bootstrapping

VideoMAP: Toward Scalable Mamba-based Video Autoregressive Pretraining

Foundation X: Integrating Classification, Localization, and Segmentation through Lock-Release Pretraining Strategy for Chest X-ray Analysis

OLMoE: Open Mixture-of-Experts Language Models

Backdooring Vision-Language Models with Out-Of-Distribution Data

Towards Semantic Equivalence of Tokenization in Multimodal LLM

DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization

CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models

Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework

Understanding Emergent Abilities of Language Models from the Loss Perspective

Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models

CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model

Failures to Find Transferable Image Jailbreaks Between Vision-Language Models

Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding

ST-Align: A Multimodal Foundation Model for Image-Gene Alignment in Spatial Transcriptomics

BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation

Why are Visually-Grounded Language Models Bad at Image Classification?

BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays

Elucidating the design space of language models for image generation

CLIP-VAD: Exploiting Vision-Language Models for Voice Activity Detection

ACER: Automatic Language Model Context Extension via Retrieval

ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation