语言大模型 / LLM

Self-Supervised Pretraining for Aerial Road Extraction

Rupert Polley, Sai Vignesh Abishek Deenadayalan, J. Marius Zöllner

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Accepted at 36th IEEE Intelligent Vehicles Symposium (IV) 2025 Joint Workshop on Safety, Metrics and Benchmarks for Autonomous Driving

PharMolixFM: All-Atom Foundation Models for Molecular Modeling and Generation

Yizhen Luo, Jiashuo Wang, Siqi Fan, Zaiqing Nie

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

GP-MoLFormer: A Foundation Model For Molecular Generation

Jerret Ross, Brian Belgodere, Samuel C. Hoffman, Vijil Chenthamarakshan, Jiri Navratil, Youssef Mroueh, Payel Das

专题命中 预训练与数据 ：foundation model(title)；language model(abstract)；分类 cs.LG

CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models

Yujian Zhao, Chengru Wu, Yinong Xu, Xuanzheng Du, Ruiyu Li, Guanglin Niu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by ICME 2025

Sample, estimate, aggregate: A recipe for causal discovery foundation models

Menghua Wu, Yujia Bao, Regina Barzilay, Tommi Jaakkola

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments Our code is available at https://github.com/rmwu/sea

Journal ref Transactions on Machine Learning Research (03/2025)

A Pretraining-Finetuning Computational Framework for Material Homogenization

Yizheng Wang, Xiang Li, Ziming Yan, Shuaifeng Ma, Jinshuai Bai, Bokai Liu, Timon Rabczuk, Yinghua Liu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Detection of Somali-written Fake News and Toxic Messages on the Social Media Using Transformer-based Language Models

Muhidin A. Mohamed, Shuab D. Ahmed, Yahye A. Isse, Hanad M. Mohamed, Fuad M. Hassan, Houssein A. Assowe

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

CLASP: Contrastive Language-Speech Pretraining for Multilingual Multimodal Information Retrieval

Mohammad Mahdi Abootorabi, Ehsaneddin Asgari

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments accepted at ECIR 2025, 13 pages, 4 figures

E2ETune: End-to-End Knob Tuning via Fine-tuned Generative Language Model

Xinmei Huang, Haoyang Li, Jing Zhang, Xinxin Zhao, Zhiming Yao, Yiyan Li, Tieying Zhang, Jianjun Chen, Hong Chen, Cuiping Li

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI

Comments Accepted by VLDB 2025

HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition

Kun Yuan, Vinkle Srivastav, Nassir Navab, Nicolas Padoy

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by MICCAI2024

Synthetic Data is an Elegant GIFT for Continual Vision-Language Models

Bin Wu, Wuxuan Shi, Jinqiao Wang, Mang Ye

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Comments This work is accepted by CVPR 2025. Modifications may be performed

TimeFound: A Foundation Model for Time Series Forecasting

Congxi Xiao, Jingbo Zhou, Yixiong Xiao, Xinjiang Lu, Le Zhang, Hui Xiong

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

DELST: Dual Entailment Learning for Hyperbolic Image-Gene Pretraining in Spatial Transcriptomics

Xulin Chen, Junzhou Huang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Unsupervised Parameter Efficient Source-free Post-pretraining

Abhishek Jha, Tinne Tuytelaars, Yuki M. Asano

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Pretraining Frequency Predicts Compositional Generalization of CLIP on Real-World Tasks

Thaddäus Wiedemer, Yash Sharma, Ameya Prabhu, Matthias Bethge, Wieland Brendel

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments NeurIPS 2024 Workshop on Compositional Learning: Perspectives, Methods, and Paths Forward

WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling

Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, Ziang Zhang, Xiaoda Yang, Rongjie Huang, Yidi Jiang, Qian Chen, Siqi Zheng, Zhou Zhao

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Comments Accepted by ICLR 2025

Governing AI Beyond the Pretraining Frontier

Nicholas A. Caputo

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments 14 pages

Object-centric Binding in Contrastive Language-Image Pretraining

Rim Assouel, Pietro Astolfi, Florian Bordes, Michal Drozdzal, Adriana Romero-Soriano

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.AI

Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?

Nishant Balepur, Feng Gu, Abhilasha Ravichander, Shi Feng, Jordan Boyd-Graber, Rachel Rudinger

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL

Comments NAACL 2025

Making Language Models Robust Against Negation

MohammadHossein Rezaei, Eduardo Blanco

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2025

Detecting Backdoor Samples in Contrastive Language Image Pretraining

Hanxun Huang, Sarah Erfani, Yige Li, Xingjun Ma, James Bailey

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments ICLR2025

Foundation Models in Radiology: What, How, When, Why and Why Not

Magdalini Paschali, Zhihong Chen, Louis Blankemeier, Maya Varma, Alaa Youssef, Christian Bluethgen, Curtis Langlotz, Sergios Gatidis, Akshay Chaudhari

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments This pre-print has been accepted for publication in Radiology. (DOI for the peer-reviewed article: 10.1148/radiol.240597)

Beyond Random Augmentations: Pretraining with Hard Views

Fabio Ferreira, Ivo Rapant, Jörg K. H. Franke, Frank Hutter

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Journal ref The Thirteenth International Conference on Learning Representations (ICLR) 2025

A generative foundation model for an all-in-one seismic processing framework

Shijun Cheng, Randy Harsuko, Tariq Alkhalifah

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI

How Does the Spatial Distribution of Pre-training Data Affect Geospatial Foundation Models?

Mirali Purohit, Gedeon Muhawenayo, Esther Rolf, Hannah Kerner

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments Accepted at Good Data for Generative AI @ AAAI 2025

UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction

Xixuan Hao, Wei Chen, Yibo Yan, Siru Zhong, Kun Wang, Qingsong Wen, Yuxuan Liang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments Accepted as a full paper by AAAI'25 - AI for Social Impact Track

On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena

Tarek Naous, Wei Xu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Evaluating Time Series Foundation Models on Noisy Periodic Time Series

Syamantak Datta Gupta

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining

Holger Severin Bovbjerg, Jan Østergaard, Jesper Jensen, Zheng-Hua Tan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Submitted to IEEE/ACM Transactions on Audio, Speech, and Language Processing for possible publication. 12 pages, 4 figures, 5 tables

Tuning Vision-Language Models with Candidate Labels by Prompt Alignment

Zhifang Zhang, Yuwei Niu, Xin Liu, Beibei Li

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI