语言大模型 / LLM

Harnessing small projectors and multiple views for efficient vision pretraining

Kumar Krishna Agrawal, Arna Ghosh, Shagun Sodhani, Adam Oberman, Blake Richards

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to NeurIPS 2024

Reducing the Sensitivity of Neural Physics Simulators to Mesh Topology via Pretraining

Nathan Vaska, Justin Goodwin, Robin Walters, Rajmonda S. Caceres

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments 5 pages, 3 figures

Investigating the Impact of Data Selection Strategies on Language Model Performance

Jiayao Gu, Liting Chen, Yihong Li

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 7 pages, 1 figure

VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset

Jing Liu, Sihan Chen, Xingjian He, Longteng Guo, Xinxin Zhu, Weining Wang, Jinhui Tang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments Preprint version w/o audio files embeded in PDF. Audio embeded version can be found on project page or github

Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models

Benjamin Icard, Evangelia Zve, Lila Sainero, Alice Breton, Jean-Gabriel Ganascia

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments To appear in the Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), Abu Dhabi

LLM for Barcodes: Generating Diverse Synthetic Data for Identity Documents

Hitesh Laxmichand Patel, Amit Agarwal, Bhargava Kumar, Karan Gupta, Priyaranjan Pattnayak

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments 5 pages, 1 figures

Enabling Time-series Foundation Model for Building Energy Forecasting via Contrastive Curriculum Learning

Rui Liang, Yang Deng, Donghua Xie, Fang He, Dan Wang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Which LLMs are Difficult to Detect? A Detailed Analysis of Potential Factors Contributing to Difficulties in LLM Text Detection

Shantanu Thorat, Tianbao Yang

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at NeurIPS 2024 - Safe Generative AI Workshop; Camera-ready version

Measuring Pre-training Data Quality without Labels for Time Series Foundation Models

Songkang Wen, Vasilii Feofanov, Jianfeng Zhang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Journal ref NeurIPS 2024 Workshop on Time Series in the Age of Large Models

Facial Wrinkle Segmentation for Cosmetic Dermatology: Pretraining with Texture Map-Based Weak Supervision

Junho Moon, Haejun Chung, Ikbeom Jang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted at International Conference on Pattern Recognition (ICPR), 2024

MPLite: Multi-Aspect Pretraining for Mining Clinical Health Records

Eric Yang, Pengfei Hu, Xiaoxue Han, Yue Ning

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language

Marcos Piau, Roberto Lotufo, Rodrigo Nogueira

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models

Yeming Wen, Swarat Chaudhuri

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

A Mamba Foundation Model for Time Series Forecasting

Haoyu Ma, Yushu Chen, Wenlai Zhao, Jinzhe Yang, Yingsheng Ji, Xinghua Xu, Xiaozhu Liu, Hao Jing, Shengzhuo Liu, Guangwen Yang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Training Compute-Optimal Protein Language Models

Xingyi Cheng, Bo Chen, Pan Li, Jing Gong, Jie Tang, Le Song

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024 (Spotlight); Code: https://github.com/cxysteven/ScalingProteinLM. Additional resources are available here

Teaching a Language Model to Distinguish Between Similar Details using a Small Adversarial Training Set

Chris Achard

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance

Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip H. S. Torr, Adel Bibi, Samuel Albanie, Matthias Bethge

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments Short version accepted at DPFM, ICLR'24; Full paper at NeurIPS'24

MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training

Bo Chen, Zhilei Bei, Xingyi Cheng, Pan Li, Jie Tang, Le Song

专题命中 预训练与数据 ：prompting(title)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024

Few-Shot Adversarial Prompt Learning on Vision-Language Models

Yiwei Zhou, Xiaobo Xia, Zhiwei Lin, Bo Han, Tongliang Liu

专题命中 预训练与数据 ：language model(title)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024

On Catastrophic Inheritance of Large Foundation Models

Hao Chen, Bhiksha Raj, Xing Xie, Jindong Wang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by DMLR

Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations

Neale Ratzlaff, Matthew Lyle Olson, Musashi Hinck, Shao-Yen Tseng, Vasudev Lal, Phillip Howard

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS workshop on SafeGenAI, 10 pages, 2 figures

Scaling Wearable Foundation Models

Girish Narayanswamy, Xin Liu, Kumar Ayush, Yuzhe Yang, Xuhai Xu, Shun Liao, Jake Garrison, Shyam Tailor, Jake Sunshine, Yun Liu, Tim Althoff, Shrikanth Narayanan, Pushmeet Kohli, Jiening Zhan, Mark Malhotra, Shwetak Patel, Samy Abdel-Ghaffar, Daniel McDuff

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging

Jacob Morrison, Noah A. Smith, Hannaneh Hajishirzi, Pang Wei Koh, Jesse Dodge, Pradeep Dasigi

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Findings of EMNLP 2024

DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences

Yidong Huang, Jacob Sansom, Ziqiao Ma, Felix Gervits, Joyce Chai

专题命中 预训练与数据 ：LLM(title)；foundation model(abstract)；分类 cs.CL、cs.AI

Comments 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models

Yuancheng Xu, Jiarui Yao, Manli Shu, Yanchao Sun, Zichu Wu, Ning Yu, Tom Goldstein, Furong Huang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by Thirty-Eighth Annual Conference on Neural Information Processing Systems (Neurips 2024)

Evaluating Copyright Takedown Methods for Language Models

Boyi Wei, Weijia Shi, Yangsibo Huang, Noah A. Smith, Chiyuan Zhang, Luke Zettlemoyer, Kai Li, Peter Henderson

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 31 pages, 9 figures, 14 tables

Pretraining Graph Transformers with Atom-in-a-Molecule Quantum Properties for Improved ADMET Modeling

Alessio Fallani, Ramil Nugmanov, Jose Arjona-Medina, Jörg Kurt Wegner, Alexandre Tkatchenko, Kostiantyn Chernichenko

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Theia: Distilling Diverse Vision Foundation Models for Robot Learning

Jinghuan Shang, Karl Schmeckpeper, Brandon B. May, Maria Vittoria Minniti, Tarik Kelestemur, David Watkins, Laura Herlant

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments CoRL 2024

Investigating Cost-Efficiency of LLM-Generated Training Data for Conversational Semantic Frame Analysis

Shiho Matta, Yin Jou Huang, Fei Cheng, Hirokazu Kiyomaru, Yugo Murawaki

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages including 4 pages of references and appendix. 7 figures

Scaling Parameter-Constrained Language Models with Quality Data

Ernie Chang, Matteo Paltenghi, Yang Li, Pin-Jie Lin, Changsheng Zhao, Patrick Huber, Zechun Liu, Rastislav Rabatin, Yangyang Shi, Vikas Chandra

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Industry Track, 18 pages, 9 figures, 4 tables