语言大模型 / LLM - arXivDaily 专题

2410.03735 2025-03-12 cs.CL cs.LG 84%

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling

David Grangier, Simin Fan, Skyler Seto, Pierre Ablin

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 23 pages, presented at the International Conference on Learning Representation (ICLR), 2025

2406.05053 2025-03-10 cs.LG cs.AI 84%

Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation

Nachiket Kotalwar, Alkis Gotovos, Adish Singla

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

2502.17910 2025-02-26 cs.CL cs.AI 84%

Scaling LLM Pre-training with Vocabulary Curriculum

Fangyuan Yu

专题命中预训练与数据：LLM(title)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments under review

2502.09642 2025-02-25 cs.CL cs.AI 84%

Krutrim LLM: Multilingual Foundational Model for over a Billion People

Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv, Deepak Kumar, Raghav Awasthi, Arveti Manjunath, Himanshu Gupta, Shubham Agarwal, Kumar Ashish, Gautam Bhargava, Chandra Khatri

专题命中预训练与数据：LLM(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.AI

2502.15466 2025-02-24 cs.LG cs.AI 84%

Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation

Wenxuan Wang, Kai Wu, Yujian Betterest Li, Dan Wang, Xiaoyu Zhang, Jing Liu

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2502.03383 2025-02-06 cs.LG cs.AI 84%

Transformers and Their Roles as Time Series Foundation Models

Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 34 Pages, 2 Figures

2501.17104 2025-01-29 cs.CL cs.AI 84%

COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Models

Tobias Materzok

专题命中预训练与数据：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2407.01492 2025-01-24 cs.CL cs.AI 84%

RegMix: Data Mixture as Regression for Language Model Pre-training

Qian Liu, Xiaosen Zheng, Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang, Min Lin

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025

2210.06433 2025-01-13 cs.CV cs.AI cs.LG 84%

Self-supervised video pretraining yields robust and more human-aligned visual representations

Nikhil Parthasarathy, S. M. Ali Eslami, João Carreira, Olivier J. Hénaff

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

2412.18863 2024-12-30 cs.CL cs.AI 84%

Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models

Meltem Aksoy

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2412.17449 2024-12-24 cs.LG cs.AI 84%

Applying LLM and Topic Modelling in Psychotherapeutic Contexts

Alexander Vanin, Vadim Bolshev, Anastasia Panfilova

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 4 figures

2412.12541 2024-12-18 cs.CL cs.AI 84%

LLMCL-GEC: Advancing Grammatical Error Correction with LLM-Driven Curriculum Learning

Tao Fang, Derek F. Wong, Lusheng Zhang, Keyan Jin, Qiang Zhang, Tianjiao Li, Jinlong Hou, Lidia S. Chao

专题命中预训练与数据：LLM(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Derek F. Wong is the corresponding author. The preprint version consists of 15 Pages, 5 Figures, 5 Tables, and 3 Appendices

2412.11477 2024-12-17 cs.LG cs.CL 84%

NoteContrast: Contrastive Language-Diagnostic Pretraining for Medical Text

Prajwal Kailas, Max Homilius, Rahul C. Deo, Calum A. MacRae

专题命中预训练与数据：pretraining(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Journal ref Proceedings of the 3rd Machine Learning for Health Symposium, PMLR 225:201-216, 2023

2412.10717 2024-12-17 cs.CL cs.AI 84%

HITgram: A Platform for Experimenting with n-gram Language Models

Shibaranjani Dasgupta, Chandan Maity, Somdip Mukherjee, Rohan Singh, Diptendu Dutta, Debasish Jana

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2407.06438 2024-12-17 cs.CV cs.CL cs.LG 84%

SOLO: A Single Transformer for Scalable Vision-Language Modeling

Yangyi Chen, Xingyao Wang, Hao Peng, Heng Ji

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to TMLR

2412.08111 2024-12-12 cs.CV cs.CL cs.LG 84%

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models

Sri Harsha Dumpala, David Arps, Sageev Oore, Laura Kallmeyer, Hassan Sajjad

专题命中预训练与数据：language model(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

2310.02994 2024-12-12 cs.LG cs.AI stat.ML 84%

Multiple Physics Pretraining for Physical Surrogate Models

Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, Mariel Pettee, Tiberiu Tesileanu, Kyunghyun Cho, Shirley Ho

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

2305.08339 2024-12-10 cs.CL cs.AI 84%

Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology

Danni Yu, Luyang Li, Hang Su, Matteo Fuoli

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 2 figures, 3 tablels

2410.15661 2024-12-10 cs.CL cs.LG 84%

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

Clara Na, Ian Magnusson, Ananya Harsh Jha, Tom Sherborne, Emma Strubell, Jesse Dodge, Pradeep Dasigi

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2024. 17 pages

2408.14471 2024-12-09 cs.CV cs.CL cs.LG 84%

A Practitioner's Guide to Continual Multimodal Pretraining

Karsten Roth, Vishaal Udandarao, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals, Olivier Hénaff, Samuel Albanie, Matthias Bethge, Zeynep Akata

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments Technical Report. 52 pages. Shorter version published at the NeurIPS 2024 Dataset & Benchmarks track

2410.12883 2024-12-05 cs.CL cs.LG 84%

Scaling Laws for Multilingual Language Models

Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishrav Chaudhary, Han Zhao, Xia Song

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2411.15099 2024-11-25 cs.CV cs.CL cs.LG 84%

Context-Aware Multimodal Pretraining

Karsten Roth, Zeynep Akata, Dima Damen, Ivana Balažević, Olivier J. Hénaff

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

2406.06046 2024-11-19 cs.CL cs.LG 84%

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models

Zichun Yu, Spandan Das, Chenyan Xiong

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024

2411.03356 2024-11-07 cs.LG cs.AI 84%

Enhancing Table Representations with LLM-powered Synthetic Data Generation

Dayu Yang, Natawut Monaikul, Amanda Ding, Bozhao Tan, Kishore Mosaliganti, Giri Iyengar

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments the Thirty-Eighth Annual Conference on Neural Information Processing Systems Table Representation Workshop

2405.17992 2024-11-05 cs.CL cs.AI q-bio.NC 84%

fMRI predictors based on language models of increasing complexity recover brain left lateralization

Laurent Bonnasse-Gahot, Christophe Pallier

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2411.00997 2024-11-05 cs.CV cs.AI cs.CL cs.CY 84%

Identifying Implicit Social Biases in Vision-Language Models

Kimia Hamidieh, Haoran Zhang, Walter Gerych, Thomas Hartvigsen, Marzyeh Ghassemi

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2410.07018 2024-11-05 cs.LG cs.AI 84%

Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization

Chengtao Jian, Kai Yang, Yang Jiao

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

2405.16528 2024-11-05 cs.LG cs.CL 84%

LoQT: Low-Rank Adapters for Quantized Pretraining

Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano, Serge Belongie, Vésteinn Snæbjarnarson

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

2410.24177 2024-11-01 eess.AS cs.CL cs.LG cs.SD 84%

DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models

Heng-Jui Chang, Hongyu Gong, Changhan Wang, James Glass, Yu-An Chung

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL、cs.LG

Comments Preprint

2410.18160 2024-10-25 cs.CL cs.LG 84%

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction

Nicholas Walker

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 15 pages, 7 figures, 3 tables

1. 预训练与数据 12393 篇

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling

Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation

Scaling LLM Pre-training with Vocabulary Curriculum

Krutrim LLM: Multilingual Foundational Model for over a Billion People

Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation

Transformers and Their Roles as Time Series Foundation Models

COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Models

RegMix: Data Mixture as Regression for Language Model Pre-training

Self-supervised video pretraining yields robust and more human-aligned visual representations

Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models

Applying LLM and Topic Modelling in Psychotherapeutic Contexts

LLMCL-GEC: Advancing Grammatical Error Correction with LLM-Driven Curriculum Learning

NoteContrast: Contrastive Language-Diagnostic Pretraining for Medical Text

HITgram: A Platform for Experimenting with n-gram Language Models

SOLO: A Single Transformer for Scalable Vision-Language Modeling

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models

Multiple Physics Pretraining for Physical Surrogate Models

Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

A Practitioner's Guide to Continual Multimodal Pretraining

Scaling Laws for Multilingual Language Models

Context-Aware Multimodal Pretraining

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models

Enhancing Table Representations with LLM-powered Synthetic Data Generation

fMRI predictors based on language models of increasing complexity recover brain left lateralization

Identifying Implicit Social Biases in Vision-Language Models

Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization

LoQT: Low-Rank Adapters for Quantized Pretraining

DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction