语言大模型 / LLM

LLM-Powered User Simulator for Recommender System

Zijian Zhang, Shuchang Liu, Ziru Liu, Rui Zhong, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Qidong Liu, Peng Jiang

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.AI

Vision-and-Language Pretraining

Thong Nguyen, Cong-Duy Nguyen, Xiaobao Wu, See-Kiong Ng, Anh Tuan Luu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments The content of the paper has been outdated. I would like to rewrite a new version with completely new information.

Why Fine-grained Labels in Pretraining Benefit Generalization?

Guan Zhe Hong, Yin Cui, Ariel Fuxman, Stanley Chan, Enming Luo

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2303.16887

When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining

Juan Yeo, Jinkwan Jang, Kyubyung Chae, Seongkyu Mun, Taesup Kim

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 5 pages, 3 figures

Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Michael Y. Hu, Aaron Mueller, Candace Ross, Adina Williams, Tal Linzen, Chengxu Zhuang, Ryan Cotterell, Leshem Choshen, Alex Warstadt, Ethan Gotlieb Wilcox

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.CL

Latent Neural Operator Pretraining for Solving Time-Dependent PDEs

Tian Wang, Chuang Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models

Theo Cachet, Christopher R. Dance, Olivier Sigaud

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training

Kaustubh Ponkshe, Venkatapathy Subramanian, Natwar Modani, Ganesh Ramakrishnan

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Solaris: A Foundation Model of the Sun

Harris Abdul Majid, Pietro Sittoni, Francesco Tudisco

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

ENCLIP: Ensembling and Clustering-Based Contrastive Language-Image Pretraining for Fashion Multimodal Search with Limited Data and Low-Quality Images

Prithviraj Purushottam Naik, Rohit Agarwal

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

HistoEncoder: a digital pathology foundation model for prostate cancer

Joona Pohjonen, Abderrahim-Oussama Batouche, Antti Rannikko, Kevin Sandeman, Andrew Erickson, Esa Pitkanen, Tuomas Mirtti

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI

CamemBERT 2.0: A Smarter French Language Model Aged to Perfection

Wissam Antoun, Francis Kulumba, Rian Touchent, Éric de la Clergerie, Benoît Sagot, Djamé Seddah

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning

Hongsheng Zhang, Zhong Ji, Jingren Liu, Yanwei Pang, Jungong Han

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

MaLa-ASR: Multimedia-Assisted LLM-Based ASR

Guanrou Yang, Ziyang Ma, Fan Yu, Zhifu Gao, Shiliang Zhang, Xie Chen

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.AI

Illuminating Blind Spots of Language Models with Targeted Agent-in-the-Loop Synthetic Data

Philip Lippmann, Matthijs T. J. Spaan, Jie Yang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Preprint. Under review

Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining

Ruoxi Cheng, Yizhong Ding, Shuirong Cao, Shitong Shao, Zhiqiang Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs

Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Berner, Raymond A. Yeh, Jean Kossaifi, Kamyar Azizzadenesheli, Anima Anandkumar

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes

Zébulon Goriely, Richard Diehl Martinez, Andrew Caines, Lisa Beinborn, Paula Buttery

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Déjà Vu Memorization in Vision-Language Models

Bargav Jayaraman, Chuan Guo, Kamalika Chaudhuri

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5

Thao Anh Dang, Limor Raviv, Lukas Galke

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments 8 pages

Spirit LM: Interleaved Spoken and Written Language Model

Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R. Costa-jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, Mary Williamson, Gabriel Synnaeve, Juan Pino, Benoit Sagot, Emmanuel Dupoux

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

PORTAL: Scalable Tabular Foundation Models via Content-Specific Tokenization

Marco Spinaci, Marek Polewczyk, Johannes Hoffart, Markus C. Kohler, Sam Thelin, Tassilo Klein

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.LG

Comments Accepted at Table Representation Learning Workshop at NeurIPS 2024

From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition

Qiyuan Yang, Pengda Wang, Luke D. Plonsky, Frederick L. Oswald, Hanjie Chen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

TMI! Finetuned Models Leak Private Information from their Pretraining Data

John Abascal, Stanley Wu, Alina Oprea, Jonathan Ullman

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Diffusion Language Models Are Versatile Protein Learners

Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, Quanquan Gu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.LG

Comments ICML 2024 camera-ready version

TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining

Ruiyi Zhang, Sai Ashish Somayajula, Pengtao Xie

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries

Supriti Vijay, Aman Priyanshu, Ashique R. KhudaBukhsh

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL

Comments 12 pages, 3 figures, 4 tables

An Efficient Contrastive Unimodal Pretraining Method for EHR Time Series Data

Ryan King, Shivesh Kodali, Conrad Krueger, Tianbao Yang, Bobak J. Mortazavi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Task-Specific Adaptation of Segmentation Foundation Model via Prompt Learning

Hyung-Il Kim, Kimin Yun, Jun-Seok Yun, Yuseok Bae

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI

Comments Workshop on OOD Generalization in Computer Vision, ECCV 2024

Cross-Task Pretraining for Cross-Organ Cross-Scanner Adenocarcinoma Segmentation

Adrian Galdran

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments MICCAI2024 COSAS Challenge - short abstract