语言大模型 / LLM

Only the Curve Shape Matters: Training Foundation Models for Zero-Shot Multivariate Time Series Forecasting through Next Curve Shape Prediction

Cheng Feng, Long Huang, Denis Krompass

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Online Foundation Model Selection in Robotics

Po-han Li, Oyku Selin Toprak, Aditya Narayanan, Ufuk Topcu, Sandeep Chinchali

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting

Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Hena Ghonia, Rishika Bhagwatkar, Arian Khorasani, Mohammad Javad Darvishi Bayazi, George Adamopoulos, Roland Riachi, Nadhir Hassen, Marin Biloš, Sahil Garg, Anderson Schneider, Nicolas Chapados, Alexandre Drouin, Valentina Zantedeschi, Yuriy Nevmyvaka, Irina Rish

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments First two authors contributed equally. All data, models and code used are open-source. GitHub: https://github.com/time-series-foundation-models/lag-llama

Prismer: A Vision-Language Model with Multi-Task Experts

Shikun Liu, Linxi Fan, Edward Johns, Zhiding Yu, Chaowei Xiao, Anima Anandkumar

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Published at TMLR 2024. Project Page: https://shikun.io/projects/prismer Code: https://github.com/NVlabs/prismer

Generative Pretraining at Scale: Transformer-Based Encoding of Transactional Behavior for Fraud Detection

Ze Yu Zhao, Zheng Zhu, Guilin Li, Wenhan Wang, Bo Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Learning Speech Representation From Contrastive Token-Acoustic Pretraining

Chunyu Qiang, Hao Li, Yixin Tian, Ruibo Fu, Tao Wang, Longbiao Wang, Jianwu Dang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICASSP 2024

Does Visual Pretraining Help End-to-End Reasoning?

Chen Sun, Calvin Luo, Xingyi Zhou, Anurag Arnab, Cordelia Schmid

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2023

ClimaX: A foundation model for weather and climate

Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, Aditya Grover

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments International Conference on Machine Learning 2023

Impact of Adversarial Training on Robustness and Generalizability of Language Models

Enes Altinisik, Hassan Sajjad, Husrev Taha Sencar, Safa Messaoud, Sanjay Chawla

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusion

Weitao Du, Jiujiu Chen, Xuecang Zhang, Zhiming Ma, Shengchao Liu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2023

Visual Data-Type Understanding does not emerge from Scaling Vision-Language Models

Vishaal Udandarao, Max F. Burg, Samuel Albanie, Matthias Bethge

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Scaling Laws for Adversarial Attacks on Language Model Activations

Stanislav Fort

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 15 pages, 9 figures

Scalable Extraction of Training Data from (Production) Language Models

Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, Katherine Lee

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Analyzing Zero-Shot Abilities of Vision-Language Models on Video Understanding Tasks

Avinash Madasu, Anahita Bhiwandiwalla, Vasudev Lal

专题命中 预训练与数据 ：language model(title)；pretraining(abstract)；分类 cs.CL、cs.AI

Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation?

Tim Hartill, Joshua Bensemann, Michael Witbrock, Patricia J. Riddle

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Can We Utilize Pre-trained Language Models within Causal Discovery Algorithms?

Chanhui Lee, Juhyeon Kim, Yongjun Jeong, Juhyun Lyu, Junghee Kim, Sangmin Lee, Sangjun Han, Hyeokjun Choe, Soyeon Park, Woohyung Lim, Sungbin Lim, Sanghack Lee

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Data Augmentation for Neural Machine Translation using Generative Language Model

Seokjin Oh, Su Ah Lee, Woohwan Jung

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

All Should Be Equal in the Eyes of Language Models: Counterfactually Aware Fair Text Generation

Pragyan Banerjee, Abhinav Java, Surgan Jandial, Simra Shahid, Shaz Furniturewala, Balaji Krishnamurthy, Sumit Bhatia

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments The first four authors contributed equally to the work

Training Socially Aligned Language Models on Simulated Social Interactions

Ruibo Liu, Ruixin Yang, Chenyan Jia, Ge Zhang, Denny Zhou, Andrew M. Dai, Diyi Yang, Soroush Vosoughi

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Code, data, and models can be downloaded via https://github.com/agi-templar/Stable-Alignment

Proposal-Contrastive Pretraining for Object Detection from Fewer Data

Quentin Bouniot, Romaric Audigier, Angélique Loesch, Amaury Habrard

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2023

Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models

Lina Conti, Guillaume Wisniewski

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP'23

ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction

Yaorui Shi, An Zhang, Enzhi Zhang, Zhiyuan Liu, Xiang Wang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model

Abhijith Chintam, Rahel Beloch, Willem Zuidema, Michael Hanna, Oskar van der Wal

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at BlackboxNLP 2023

Learn Your Tokens: Word-Pooled Tokenization for Language Modeling

Avijit Thawani, Saurabh Ghanekar, Xiaoyuan Zhu, Jay Pujara

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2023 Findings

Difference-Masking: Choosing What to Mask in Continued Pretraining

Alex Wilf, Syeda Nahida Akter, Leena Mathur, Paul Pu Liang, Sheryl Mathew, Mengrou Shou, Eric Nyberg, Louis-Philippe Morency

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Memorization for Good: Encryption with Autoregressive Language Models

Samuel Stevens, Yu Su

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Main text: 9 pages, 4 figures, 1 table. Work-in-progress. Project website at https://samuelstevens.me/research/encryption/

P5: Plug-and-Play Persona Prompting for Personalized Response Selection

Joosung Lee, Minsik Oh, Donghun Lee

专题命中 预训练与数据 ：prompting(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2023 main conference

On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks

Laura Fee Nern, Harsh Raj, Maurice Georgi, Yash Sharma

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

No Offense Taken: Eliciting Offensiveness from Language Models

Anugya Srivastava, Rahul Ahuja, Rohith Mukku

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Forgetting Private Textual Sequences in Language Models via Leave-One-Out Ensemble

Zhe Liu, Ozlem Kalinli

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI