语言大模型 / LLM

ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models

Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, Benyou Wang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 22 pages

Language models scale reliably with over-training and on downstream tasks

Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, Suchin Gururangan, Mitchell Wortsman, Rulin Shao, Jean Mercat, Alex Fang, Jeffrey Li, Sedrick Keh, Rui Xin, Marianna Nezhurina, Igor Vasiljevic, Jenia Jitsev, Luca Soldaini, Alexandros G. Dimakis, Gabriel Ilharco, Pang Wei Koh, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, Ludwig Schmidt

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions

Anfeng Xu, Kevin Huang, Tiantian Feng, Lue Shen, Helen Tager-Flusberg, Shrikanth Narayanan

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.CL、cs.LG

Comments Interspeech 2024

Multi-Modal Automatic Prosody Annotation with Contrastive Pretraining of SSWP

Jinzuomu Zhong, Yang Li, Hui Huang, Korin Richmond, Jie Liu, Zhiba Su, Jing Guo, Benlai Tang, Fengjie Zhu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

An Independence-promoting Loss for Music Generation with Language Models

Jean-Marie Lemercier, Simon Rouard, Jade Copet, Yossi Adi, Alexandre Défossez

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to ICML 2024

Model-Based Reinforcement Learning with Multi-Task Offline Pretraining

Minting Pan, Yitao Zheng, Yunbo Wang, Xiaokang Yang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

FOCUS: Forging Originality through Contrastive Use in Self-Plagiarism for Language Models

Kaixin Lan, Tao Fang, Derek F. Wong, Yabo Xu, Lidia S. Chao, Cecilia G. Zhao

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 8 figures. The paper has been accepted by ACL 2024 (Findings), with Kaixin Lan and Tao Fang contributing equally, and Derek F. Wong serving as the corresponding author

Cross-Table Pretraining towards a Universal Function Space for Heterogeneous Tabular Data

Jintai Chen, Zhen Lin, Qiyuan Chen, Jimeng Sun

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning

Sheng Yue, Xingyuan Hua, Ju Ren, Sen Lin, Junshan Zhang, Yaoxue Zhang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments International Conference on Machine Learning (ICML)

Improving Speech Decoding from ECoG with Self-Supervised Pretraining

Brian A. Yuan, Joseph G. Makin

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm

Zhanpeng Zhou, Zijun Chen, Yilan Chen, Bo Zhang, Junchi Yan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments 31 pages, 24 figures

Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss

Ruijie Zheng, Yongyuan Liang, Xiyao Wang, Shuang Ma, Hal Daumé, Huazhe Xu, John Langford, Praveen Palanisamy, Kalyan Shankar Basu, Furong Huang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted at Forty-first International Conference on Machine Learning (ICML 2024)

Continued Pretraining for Domain Adaptation of Wav2vec2.0 in Automatic Speech Recognition for Elementary Math Classroom Settings

Ahmed Adel Attia, Dorottya Demszky, Tolulope Ogunremi, Jing Liu, Carol Espy-Wilson

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

A Systematic Analysis on the Temporal Generalization of Language Models in Social Media

Asahi Ushio, Jose Camacho-Collados

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulation

Jared Mejia, Victoria Dean, Tess Hellebrekers, Abhinav Gupta

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to ICRA 2024

Exploring prompts to elicit memorization in masked language model-based named entity recognition

Yuxi Xia, Anastasiia Sedova, Pedro Henrique Luz de Araujo, Vasiliki Kougia, Lisa Nußbaumer, Benjamin Roth

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Introducing cosmosGPT: Monolingual Training for Turkish Language Models

H. Toprak Kesgin, M. Kaan Yuce, Eren Dogan, M. Egemen Uzun, Atahan Uz, H. Emre Seyrek, Ahmed Zeer, M. Fatih Amasyali

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Feature Distribution Shift Mitigation with Contrastive Pretraining for Intrusion Detection

Weixing Wang, Haojin Yang, Christoph Meinel, Hasan Yagiz Özkan, Cristian Bermudez Serna, Carmen Mas-Machuca

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments accepted by ICMLCN24

Frequency-Aware Masked Autoencoders for Multimodal Pretraining on Biosignals

Ran Liu, Ellen L. Zippi, Hadi Pouransari, Chris Sandino, Jingping Nie, Hanlin Goh, Erdrin Azemi, Ali Moin

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Extended version of ICLR 2024 Learning from Time Series for Health workshop

Localizing Paragraph Memorization in Language Models

Niklas Stoehr, Mitchell Gordon, Chiyuan Zhang, Owen Lewis

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Initial Decoding with Minimally Augmented Language Model for Improved Lattice Rescoring in Low Resource ASR

Savitha Murthy, Dinkar Sitaram

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 14 pages, 7 figures, Accepted in Sadhana Journal

Towards In-Vehicle Multi-Task Facial Attribute Recognition: Investigating Synthetic Data and Vision Foundation Models

Esmaeil Seraj, Walter Talamonti

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments Manuscript under peer review

Blacks is to Anger as Whites is to Joy? Understanding Latent Affective Bias in Large Pre-trained Neural Language Models

Anoop Kadan, Deepak P., Sahely Bhadra, Manjary P. Gangan, Lajish V. L

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Mass-Producing Failures of Multimodal Systems with Language Models

Shengbang Tong, Erik Jones, Jacob Steinhardt

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Under Review

Contextual Text Denoising with Masked Language Models

Yifu Sun, Haoming Jiang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models

Lin Li, Haoyan Guan, Jianing Qiu, Michael Spratling

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments CVPR2024

Label Informed Contrastive Pretraining for Node Importance Estimation on Knowledge Graphs

Tianyu Zhang, Chengbin Hou, Rui Jiang, Xuegong Zhang, Chenghu Zhou, Ke Tang, Hairong Lv

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by IEEE TNNLS

Learning Translations: Emergent Communication Pretraining for Cooperative Language Acquisition

Dylan Cope, Peter McBurney

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Making Pre-trained Language Models Better Continual Few-Shot Relation Extractors

Shengkun Ma, Jiale Han, Yi Liang, Bo Cheng

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as COLING2024

CEV-LM: Controlled Edit Vector Language Model for Shaping Natural Language Generations

Samraj Moorjani, Adit Krishnan, Hari Sundaram

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 16 pages, 3 figures, accepted into EACL 2024