语言大模型 / LLM

Model Inversion Attacks on Llama 3: Extracting PII from Large Language Models

Sathesh P. Sivashanmugam

机构 * Senior Software Engineer American Express（美国运通高级软件工程师） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling

Tianhao Chen, Xin Xu, Zijing Liu, Pengxiang Li, Xinyuan Song, Ajay Kumar Jaiswal, Fan Zhang, Jishan Hu, Yang Wang, Hao Chen, Shizhe Diao, Shiwei Liu, Yu Li, Lu Yin, Can Yang

机构 * The Hong Kong University of Science and Technology（香港科技大学） ； International Digital Economy Academy（国际数字经济学院） ； Dalian University of Technology（大连理工大学） ； Emory University（埃默里大学） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） ； NVIDIA（英伟达） ； University of Oxford（牛津大学） ； University of Surrey（萨里大学） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation

Deyu Zou, Yongqiang Chen, Mufei Li, Siqi Miao, Chenxi Liu, Bo Han, James Cheng, Pan Li

机构 * The Chinese University of Hong Kong（香港中文大学） ； Georgia Institute of Technology（佐治亚理工学院） ； Hong Kong Baptist University（香港 Baptist大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Models

Zhicheng Zhang, Ziyan Wang, Yali Du, Fei Fang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Under Review

Is There a Case for Conversation Optimized Tokenizers in Large Language Models?

Raquel Ferrando, Javier Conde, Gonzalo Martínez, Pedro Reviriego

机构 * ETSI de Telecomunicación Universidad Politécnica de Madrid（电信工程学院马德里理工大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

GenDFIR: Advancing Cyber Incident Timeline Analysis Through Retrieval Augmented Generation and Large Language Models

Fatma Yasmine Loumachi, Mohamed Chahine Ghanem, Mohamed Amine Ferrag

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 24 pages V5.3

AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling

Yuheng Huang, Jiayang Song, Qiang Hu, Felix Juefei-Xu, Lei Ma

机构 * The University of Tokyo（东京大学） ； University of Alberta（阿尔伯塔大学） ； Tianjin University（天津大学） ； New York University（纽约大学） ； The University of Tokyo, Japan（东京大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments To appear in ACM Transactions on Software Engineering and Methodology (2025)

TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining

Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, Fartash Faghri

机构 * University of Washington（华盛顿大学） ； Apple（苹果公司） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Comments Code available at: https://github.com/apple/ml-tic-lm

SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition

Shuangrui Ding, Zihan Liu, Xiaoyi Dong, Pan Zhang, Rui Qian, Junhao Huang, Conghui He, Dahua Lin, Jiaqi Wang

机构 * The Chinese University of Hong Kong（中国香港大学） ； Beihang University（北航） ； Shanghai AI Laboratory（上海人工智能实验室） ； CPII under InnoHK（创新香港下的CPII） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 main. project page: https://pjlab-songcomposer.github.io/ code: https://github.com/pjlab-songcomposer/songcomposer

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Wanyun Xie, Francesco Tonin, Volkan Cevher

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments ICML 2025

FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation

Shaolin Zhu, Tianyu Dong, Bo Li, Deyi Xiong

机构 * College of Intelligence and Computing, Tianjin University（智能与计算学院，天津大学） ； School of Software, Tsinghua University（软件学院，清华大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Neural Thermodynamic Laws for Large Language Model Training

Ziming Liu, Yizhou Liu, Jeff Gore, Max Tegmark

机构 * Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 10 figures

Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning

Yoichi Ishibashi, Taro Yano, Masafumi Oyamada

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Advanced Crash Causation Analysis for Freeway Safety: A Large Language Model Approach to Identifying Key Contributing Factors

Ahmed S. Abdelrahman, Mohamed Abdel-Aty, Samgyu Yang, Abdulrahman Faden

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration

Kexin Ding, Mu Zhou, Akshay Chaudhari, Shaoting Zhang, Dimitris N. Metaxas

机构 * Rutgers University（罗格斯大学） ； Stanford University（斯坦福大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Distilling and exploiting quantitative insights from Large Language Models for enhanced Bayesian optimization of chemical reactions

Roshan Patel, Saeed Moayedpour, Louis De Lescure, Lorenzo Kogler-Anele, Alan Cherney, Sven Jager, Yasser Jangjou

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.AI、cs.LG

DataMan: Data Manager for Pre-training Large Language Models

Ru Peng, Kexin Yang, Yawen Zeng, Junyang Lin, Dayiheng Liu, Junbo Zhao

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments ICLR2025 paper

SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models

Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, Rahul Gupta

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Are Large Language Models Memorizing Bug Benchmarks?

Daniel Ramos, Claudia Mamede, Kush Jain, Paulo Canelas, Catarina Gamboa, Claire Le Goues

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models

Lynn Chua, Badih Ghazi, Yangsibo Huang, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Amer Sinha, Chulin Xie, Chiyuan Zhang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

How Does Code Pretraining Affect Language Model Task Performance?

Jackson Petty, Sjoerd van Steenkiste, Tal Linzen

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Journal ref Transactions on Machine Learning Research (January 2025)

Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction

Sarah Ball, Simeon Allmendinger, Frauke Kreuter, Niklas Kühl

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

LLM Pretraining with Continuous Concepts

Jihoon Tack, Jack Lanchantin, Jane Yu, Andrew Cohen, Ilia Kulikov, Janice Lan, Shibo Hao, Yuandong Tian, Jason Weston, Xian Li

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Beam Prediction based on Large Language Models

Yucheng Sheng, Kai Huang, Le Liang, Peng Liu, Shi Jin, Geoffrey Ye Li

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Large Language Models are Few-shot Multivariate Time Series Classifiers

Yakun Chen, Zihao Li, Chao Yang, Xianzhi Wang, Guandong Xu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Synthetic User Behavior Sequence Generation with Large Language Models for Smart Homes

Zhiyao Xu, Dan Zhao, Qingsong Zou, Jingyu Xiao, Yong Jiang, Zhenhui Yuan, Qing Li

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Phase Transitions in Large Language Models and the $O(N)$ Model

Youran Sun, Babak Haghighat

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models

Akash Bonagiri, Lucen Li, Rajvardhan Oak, Zeerak Babar, Magdalena Wojcieszak, Anshuman Chhabra

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments This paper is in submission and under peer review

Do Large Language Models Mirror Cognitive Language Processing?

Yuqi Ren, Renren Jin, Tongxuan Zhang, Deyi Xiong

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Analyzing Memorization in Large Language Models through the Lens of Model Attribution

Tarun Ram Menta, Susmit Agrawal, Chirag Agarwal

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG