语言大模型 / LLM

Dark LLMs: The Growing Threat of Unaligned AI Models

Michael Fire, Yitzhak Elbazis, Adi Wasenstein, Lior Rokach

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

CLTP: Contrastive Language-Tactile Pre-training for 3D Contact Geometry Understanding

Wenxuan Ma, Xiaoge Cao, Yixiang Zhang, Chaofan Zhang, Shaobo Yang, Peng Hao, Bin Fang, Yinghao Cai, Shaowei Cui, Shuo Wang

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； Beihang University（北航） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； Samsung Research China（三星中国研究院） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 16 pages

Kimi-Audio Technical Report

KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, Jun Chen, Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guokun Lai, Qingcheng Li, Yangyang Liu, Weidong Sun, Jianzhou Wang, Yuzhi Wang, Yuefeng Wu, Yuxin Wu, Dongchao Yang, Hao Yang, Ying Yang, Zhilin Yang, Aoxiong Yin, Ruibin Yuan, Yutong Zhang, Zaida Zhou

机构 * Kimi Team（Kimi团队） 纠错

专题命中 预训练与数据 ：LLM(abstract)；foundation model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments

Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, Pengfei Liu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Scaling Laws for Predicting Downstream Performance in LLMs

Yangyi Chen, Binxuan Huang, Yifan Gao, Zhengyang Wang, Jingfeng Yang, Heng Ji

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to TMLR

Fast Training Dataset Attribution via In-Context Learning

Milad Fotouhi, Mohammad Taha Bahadori, Oluwaseyi Feyisetan, Payman Arabshahi, David Heckerman

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

Kairong Luo, Haodong Wen, Shengding Hu, Zhenbo Sun, Zhiyuan Liu, Maosong Sun, Kaifeng Lyu, Wenguang Chen

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Does Training on Synthetic Data Make Models Less Robust?

Lingze Zhang, Ellie Pavlick

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees

William Fleshman, Aleem Khan, Marc Marone, Benjamin Van Durme

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments In Proceedings of the Conference on Applied Machine Learning in Information Security, 2024

From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning

Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL 2025

Enhancing Knowledge Retrieval with In-Context Learning and Semantic Search through Generative AI

Mohammed-Khalil Ghali, Abdelrahman Farrag, Daehan Won, Yu Jin

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration

Thomas Walshe, Sae Young Moon, Chunyang Xiao, Yawwani Gunawardana, Fran Silavong

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 11 pages, 1 figure

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code

Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T Stillerman, Felix Friedrich, Prateek Yadav, Tanmay Laud, Vu Minh Chien, Terry Yue Zhuo, Diganta Misra, Ben Bogin, Xuan-Son Vu, Marzena Karpinska, Arnav Varma Dantuluri, Wojciech Kusa, Tommaso Furlanello, Rio Yokota, Niklas Muennighoff, Suhas Pai, Tosin Adewumi, Veronika Laippala, Xiaozhe Yao, Adalberto Junior, Alpay Ariyak, Aleksandr Drozd, Jordan Clive, Kshitij Gupta, Liangyu Chen, Qi Sun, Ken Tsui, Noah Persaud, Nour Fahmy, Tianlong Chen, Mohit Bansal, Nicolo Monti, Tai Dang, Ziyang Luo, Tien-Tung Bui, Roberto Navigli, Virendra Mehta, Matthew Blumberg, Victor May, Huu Nguyen, Sampo Pyysalo

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Preprint

LLaVA-SLT: Visual Language Tuning for Sign Language Translation

Han Liang, Chengyu Huang, Yuecheng Xu, Cheng Tang, Weicai Ye, Juze Zhang, Xin Chen, Jingyi Yu, Lan Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data

Johannes Treutlein, Dami Choi, Jan Betley, Samuel Marks, Cem Anil, Roger Grosse, Owain Evans

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024. 10 pages, 8 figures

Is Programming by Example solved by LLMs?

Wen-Ding Li, Kevin Ellis

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

SecEncoder: Logs are All You Need in Security

Muhammed Fatih Bulut, Yingqi Liu, Naveed Ahmad, Maximilian Turner, Sami Ait Ouahmane, Cameron Andrews, Lloyd Greenwald

专题命中 预训练与数据 ：language model(abstract)；small language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

A Multilingual Sentiment Lexicon for Low-Resource Language Translation using Large Languages Models and Explainable AI

Melusi Malinga, Isaac Lupanda, Mike Wa Nkongolo, Phil van Deventer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This work is part of a PhD proposal in Information Technology at the University of Pretoria, supervised by Dr. Mike Wa Nkongolo and co-supervised by Dr. Phil van Deventer, under the Low-Resource Language Processing Lab in the Department of Informatics

Extracting Unlearned Information from LLMs with Activation Steering

Atakan Seyitoğlu, Aleksei Kuvshinov, Leo Schwinn, Stephan Günnemann

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024 Workshop Safe Generative AI

A Synthetic Dataset for Personal Attribute Inference

Hanna Yukhymenko, Robin Staab, Mark Vero, Martin Vechev

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning

Xiaochuan Li, Zichun Yu, Chenyan Xiong

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Codes and data are open-sourced at https://github.com/cxcscmu/Montessori-Instruct

Can LLMs Reliably Simulate Human Learner Actions? A Simulation Authoring Framework for Open-Ended Learning Environments

Amogh Mannekote, Adam Davies, Jina Kang, Kristy Elizabeth Boyer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages, code available at https://github.com/ahans30/Binoculars

Aligners: Decoupling LLMs and Alignment

Lilian Ngweta, Mayank Agarwal, Subha Maity, Alex Gittens, Yuekai Sun, Mikhail Yurochkin

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Short version accepted as a Tiny Paper at the International Conference on Learning Representations (ICLR) 2024. Long version accepted to the Conference on Empirical Methods in Natural Language Processing (EMNLP) 2024 Findings

HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data

Hossein Hajipour, Lea Schönherr, Thorsten Holz, Mario Fritz

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 24 pages, 16 tables, 8 figures

OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving

Julong Wei, Shanshuai Yuan, Pengfei Li, Qingda Hu, Zhongxue Gan, Wenchao Ding

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Better Alignment with Instruction Back-and-Forth Translation

Thao Nguyen, Jeffrey Li, Sewoong Oh, Ludwig Schmidt, Jason Weston, Luke Zettlemoyer, Xian Li

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AgentInstruct: Toward Generative Teaching with Agentic Flows

Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-ge Chen, Olga Vrousgos, Corby Rosset, Fillipe Silva, Hamed Khanpour, Yash Lara, Ahmed Awadallah

专题命中 预训练与数据 ：language model(abstract)；instruction tuning(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation

Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, Zehuan Yuan

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments Codes and models: \url{https://github.com/FoundationVision/LlamaGen}

MultiLegalPile: A 689GB Multilingual Legal Corpus

Joel Niklaus, Veton Matoshi, Matthias Stürmer, Ilias Chalkidis, Daniel E. Ho

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2024