语言大模型 / LLM

Which Demographics do LLMs Default to During Annotation?

Johannes Schäfer, Aidan Combs, Christopher Bagdon, Jiahui Li, Nadine Probol, Lynn Greschner, Sean Papay, Yarik Menchaca Resendiz, Aswathy Velutharambath, Amelie Wührl, Sabine Weber, Roman Klinger

机构 * Fundamentals of Natural Language Processing, University of Bamberg, Germany（自然语言处理基础，巴姆贝格大学，德国） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments ACL 2025

The Prompt is Mightier than the Example

Shengzhe Xu, Nikhil Muralidhar, Naren Ramakrishnan

机构 * Computer Science Virginia Tech（计算机科学维吉尼亚理工大学） ； Computer Science Stevens Institute of Technology（计算机科学史蒂文斯理工学院） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworks

Xinyu Wang, Jinbo Bi, Minghu Song

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 17 pages, 5 main figures, 2 appendix figures. Submitted to ICML 2025

Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report

Paul Kassianik, Baturay Saglam, Alexander Chen, Blaine Nelson, Anu Vellore, Massimo Aufiero, Fraser Burch, Dhruv Kedia, Avi Zohary, Sajana Weerawardhena, Aman Priyanshu, Adam Swanda, Amy Chang, Hyrum Anderson, Kojin Oshiba, Omar Santos, Yaron Singer, Amin Karbasi

机构 * Foundation AI – Cisco Systems Inc.（基础AI – 奇思科技公司） ； Yale University（耶鲁大学） ； Security & Trust Organization – Cisco Systems Inc.（安全与信任组织 – 奇思科技公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

The Synthetic Imputation Approach: Generating Optimal Synthetic Texts For Underrepresented Categories In Supervised Classification Tasks

Joan C. Timoneda

机构 * Purdue University（普渡大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

a1: Steep Test-time Scaling Law via Environment Augmented Generation

Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, Yuyao Ge, Jun Wan, Yurong Wu, Xueqi Cheng

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling

Lechao Xiao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 25 pages

Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?

Marika Swanberg, Ryan McKenna, Edo Roth, Albert Cheu, Peter Kairouz

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs

Tamzeed Mahfuz, Satak Kumar Dey, Ruwad Naswan, Hasnaen Adil, Khondker Salman Sayeed, Haz Sameen Shahgir

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

ConceptSearch: Towards Efficient Program Search Using LLMs for Abstraction and Reasoning Corpus (ARC)

Kartik Singhal, Gautam Shroff

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Pre-print of paper accepted at AAAI 2025

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs

Juhao Liang, Zhenyang Cai, Jianqing Zhu, Huang Huang, Kewei Zong, Bang An, Mosen Alharthi, Juncai He, Lian Zhang, Haizhou Li, Benyou Wang, Jinchao Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments Accepted to NeurIPS 2024 main conference. see https://github.com/FreedomIntelligence/AceGPT-v2

RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process

Peiran Wang, Xiaogeng Liu, Chaowei Xiao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Theoretical Analysis of Byte-Pair Encoding

László Kozma, Johannes Voderholzer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Persistent Pre-Training Poisoning of LLMs

Yiming Zhang, Javier Rando, Ivan Evtimov, Jianfeng Chi, Eric Michael Smith, Nicholas Carlini, Florian Tramèr, Daphne Ippolito

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；post-training(abstract)；SFT(abstract)

Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?

Tannon Kew, Florian Schottmann, Rico Sennrich

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；instruction tuning(abstract)；pretraining(abstract)

Comments Accepted at Findings of EMNLP 2024

Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale

Wenzhen Zheng, Wenbo Pan, Xu Xu, Libo Qin, Li Yue, Ming Zhou

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 8 pages. Accepted at EMNLP 2024

X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment

Dongjae Shin, Hyeonseok Lim, Inho Won, Changsu Choi, Minjun Kim, Seungwoo Song, Hangyeol Yoo, Sangmin Kim, Kyungtae Lim

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Journal ref https://aclanthology.org/2024.findings-naacl.158

Are Bigger Encoders Always Better in Vision Large Models?

Bozhou Li, Hao Liang, Zimo Meng, Wentao Zhang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

On Machine Learning Approaches for Protein-Ligand Binding Affinity Prediction

Nikolai Schapin, Carles Navarro, Albert Bou, Gianni De Fabritiis

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 20 pages, 14 figures, 1 table

Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer

Hele-Andra Kuulmets, Taido Purason, Agnes Luhtaru, Mark Fishel

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Journal ref Findings of the Association for Computational Linguistics: NAACL 2024, pages 3309-3325

Efficient Continual Pre-training by Mitigating the Stability Gap

Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao, Yikang Shen

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps

Dicong Qiu, Wenzong Ma, Zhenfu Pan, Hui Xiong, Junwei Liang

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；SFT(abstract)

Comments Open-vocabulary, Mobile Manipulation, Dynamic Environments, 3D Semantic Maps, Zero-shot, LLMs, VLMs, 18 pages, 2 figures

Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI

Wei-Bang Jiang, Li-Ming Zhao, Bao-Liang Lu

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；prompting(abstract)

Comments The Twelfth International Conference on Learning Representations

Journal ref The Twelfth International Conference on Learning Representations, 2024

Position: Foundation Agents as the Paradigm Shift for Decision Making

Xiaoqian Liu, Xingzhou Lou, Jianbin Jiao, Junge Zhang

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；pretraining(abstract)

Comments 17 pages, camera-ready version of ICML 2024

Seeds of Stereotypes: A Large-Scale Textual Analysis of Race and Gender Associations with Diseases in Online Sources

Lasse Hyldig Hansen, Nikolaj Andersen, Jack Gallifant, Liam G. McCoy, James K Stone, Nura Izath, Marcela Aguirre-Jerez, Danielle S Bitterman, Judy Gichoya, Leo Anthony Celi

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax

Aaron Mueller, Albert Webson, Jackson Petty, Tal Linzen

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to NAACL 2024

CoDa: Constrained Generation based Data Augmentation for Low-Resource NLP

Chandra Kiran Reddy Evuru, Sreyan Ghosh, Sonal Kumar, Ramaneswaran S, Utkarsh Tyagi, Dinesh Manocha

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to NAACL 2024 Findings

InstructDET: Diversifying Referring Object Detection with Generalized Instructions

Ronghao Dang, Jiangyan Feng, Haodong Zhang, Chongjian Ge, Lin Song, Lijun Gong, Chengju Liu, Qijun Chen, Feng Zhu, Rui Zhao, Yibing Song

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments 29 pages (include Appendix) Published in ICLR

Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages

Michael Andersland

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)