大模型对齐与安全 - arXivDaily 专题

2509.11816 2025-11-14 cs.LG cs.AI cs.CL 67%

Collapse of Irrelevant Representations (CIR) Ensures Robust and Non-Disruptive LLM Unlearning

Filip Sondej, Yushi Yang

机构 * Jagiellonian University（杰尔乔夫大学）； University of Oxford（牛津大学）

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2511.01689 2025-11-04 cs.CL cs.AI cs.LG 67%

Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI

Sharan Maiya, Henning Bartsch, Nathan Lambert, Evan Hubinger

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 pages, 6 figures, 4 tables

2510.09781 2025-10-14 cs.LG cs.AI cs.CL 67%

Building a Foundational Guardrail for General Agentic Systems via Synthetic Data

Yue Huang, Hang Hua, Yujun Zhou, Pengcheng Jing, Manish Nagireddy, Inkit Padhi, Greta Dolcetti, Zhangchen Xu, Subhajit Chaudhury, Ambrish Rawat, Liubov Nedoshivina, Pin-Yu Chen, Prasanna Sattigeri, Xiangliang Zhang

机构 * University of Notre Dame（诺特大学）； MIT-IBM Watson AI Lab（MIT-IBM Watson AI实验室）； University of Washington（华盛顿大学）； Ca’ Foscari University of Venice（威尼斯卡福尔学院）； IBM Research（IBM研究院）

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2508.06944 2025-10-13 cs.LG cs.AI cs.CL cs.CV 67%

AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal Imitation-Exploration Balance

Lixuan He, Jie Feng, Yong Li

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The paper is currently under investigation regarding concerns of potential academic misconduct. While the investigation is ongoing, the authors have voluntarily requested to withdraw the manuscript

2510.05194 2025-10-08 q-bio.QM 67%

Reinforcement Learning for Clinical Reasoning: Aligning LLMs with ACR Imaging Appropriateness Criteria

Anni Tziakouri, Filippo Menolascina

专题命中安全训练：alignment(abstract)；trustworthy(abstract)

2507.17746 2025-10-06 cs.LG cs.AI cs.CL 67%

Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains

Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, Sean Hendryx

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments preprint

2509.20370 2025-09-26 cs.AI cs.CY cs.LG 67%

Philosophy-informed Machine Learning

MZ Naser

专题命中安全训练：alignment(abstract)；分类 cs.AI、cs.CY、cs.LG

2507.13255 2025-09-24 cs.CL cs.AI cs.IR cs.LG cs.MM 67%

Automating Steering for Safe Multimodal Large Language Models

Lyucheng Wu, Mengru Wang, Ziwen Xu, Tri Cao, Nay Oo, Bryan Hooi, Shumin Deng

机构 * Zhejiang University（浙江大学）； Zhejiang University - Ant Group Joint Lab of Knowledge Graph（浙江大学-蚂蚁集团知识图谱联合实验室）； National University of Singapore, NUS-NCS Joint Lab（新加坡国立大学NUS-NCS联合实验室）

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2025 Main Conference. 23 pages (8+ for main); 25 figures; 1 table

2507.03293 2025-09-24 cs.AI cs.CL cs.LG cs.SY eess.SY 67%

LogicGuard: Improving Embodied LLM agents through Temporal Logic based Critics

Anand Gokhale, Vaibhav Srivastava, Francesco Bullo

机构 * Department of Mechanical Engineering, University of California at Santa Barbara（加州大学圣芭芭拉分校机械工程系）； Department of Electrical and Computer Engineering, Michigan State University（密歇根州立大学电气与计算机工程系）

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Modified version of prior LTLCrit work with new robotics dataset

2509.13760 2025-09-18 cs.CV 67%

Iterative Prompt Refinement for Safer Text-to-Image Generation

Jinwoo Jeon, JunHyeok Oh, Hayeong Lee, Byung-Jun Lee

机构 * Korea University（韩国大学）

专题命中安全训练：alignment(abstract)；safety(abstract)

2503.16833 2025-09-10 cs.SD cs.AI cs.CL cs.CY eess.AS 67%

The Model Hears You: Audio Language Model Deployments Should Consider the Principle of Least Privilege

Luxi He, Xiangyu Qi, Michel Liao, Inyoung Cheong, Prateek Mittal, Danqi Chen, Peter Henderson

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Published at AIES 2025

2507.11966 2025-07-17 cs.CL cs.AI cs.CY 67%

Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation

Ziyu Ge, Gabriel Chua, Leanne Tan, Roy Ka-Wei Lee

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

2507.11515 2025-07-16 cs.LG cs.AI cs.CL 67%

AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air

Shiyi Yang, Xiaoxue Yu, Rongpeng Li, Jianhang Zhu, Zhifeng Zhao, Honggang Zhang

机构 * College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）； Zhejiang Lab（浙江实验室）

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 11 pages, 8 figures

2503.09639 2025-07-15 cs.MA cs.AI cs.CL cs.CY cs.HC 67%

Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy

Abe Bohan Hou, Hongru Du, Yichen Wang, Jingyu Zhang, Zixiao Wang, Paul Pu Liang, Daniel Khashabi, Lauren Gardner, Tianxing He

机构 * Department of Computer Science, Johns Hopkins University（约翰霍普金斯大学计算机科学系）； Institute of Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）； Shanghai Qi Zhi Institute（上海启智研究院）； Civil & Systems Engineering, Johns Hopkins University（约翰霍普金斯大学土木与系统工程系）； Department of Computer Science, University of Chicago（芝加哥大学计算机科学系）； Department of Epidemiology, Harvard University（哈佛大学流行病学系）； MIT Media Lab and Department of EECS, Massachusetts Institute of Technology（麻省理工学院媒体实验室和电子工程与计算机科学系）

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Accepted to COLM 2025

2507.07938 2025-07-11 cs.MM 67%

Multimodal Framework for Explainable Autonomous Driving: Integrating Video, Sensor, and Textual Data for Enhanced Decision-Making and Transparency

Abolfazl Zarghani, Amirhossein Ebrahimi, Amir Malekesfandiari

专题命中安全训练：safety(abstract)；trustworthy(abstract)

2507.06734 2025-07-10 cs.HC cs.AI cs.CL cs.CY 67%

Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool

Milena Pustet, Elisabeth Steffen, Helena Mihaljević, Grischa Stanjek, Yannis Illies

机构 * HTW Berlin（柏林应用技术大学）

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.CY

2412.06845 2025-07-08 cs.CL cs.AI cs.LG 67%

7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement

Pu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung-En Chang, Arash Akbari, Timothy Rupprecht, Lei Lu, Enfu Nan, Changdi Yang, Yumei He, Weiyan Shi, Xingchen Xu, Yu Huang, Wei Jiang, Wei Wang, Yue Chen, Yong He, Yanzhi Wang

机构 * Northeastern University（东北大学）； Harvard University（哈佛大学）； Cornell University（康奈尔大学）； Tulane University（路易斯安那州立大学）； University of Washington（华盛顿大学）； Futurewei Technologies（未来科技）； AIBAO LLC

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2507.00829 2025-07-02 cs.CR 67%

On the Surprising Efficacy of LLMs for Penetration-Testing

Andreas Happe, Jürgen Cito

专题命中安全训练：alignment(abstract)；safety(abstract)

2506.18325 2025-06-24 cs.CV 67%

NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generation

Yu Xie, Chengjie Zeng, Lingyun Zhang, Yanwei Fu

机构 * Purple Mountain Laboratories（紫金山实验室）； Fudan University（复旦大学）

专题命中安全训练：safety(abstract)；jailbreak(abstract)

2410.13284 2025-06-23 cs.CL cs.AI cs.LG 67%

Learning to Route LLMs with Confidence Tokens

Yu-Neng Chuang, Prathusha Kameswara Sarma, Parikshit Gopalan, John Boccio, Sara Bolouki, Xia Hu, Helen Zhou

机构 * Rice University（里士大学）； Apple, Inc.（苹果公司）

专题命中安全训练：trustworthy(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.13458 2025-05-29 cs.CL cs.AI cs.CR cs.LG 67%

ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails

Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, Muhao Chen

机构 * University of California, Davis（加州大学戴维斯分校）； University of Southern California（南加州大学）

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2025

2505.17714 2025-05-26 cs.LG cs.AI cs.CL 67%

PPO-BR: Dual-Signal Entropy-Reward Adaptation for Trust Region Policy Optimization

Ben Rahman

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This manuscript builds upon an earlier version posted to TechRxiv. This arXiv version includes an updated comparison with GRPO (Group Relative Policy Optimization)

2404.14397 2025-05-05 cs.CL cs.CY cs.LG 67%

RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?

Adrian de Wynter, Ishaan Watts, Tua Wongsangaroonsri, Minghui Zhang, Noura Farra, Nektar Ege Altıntoprak, Lena Baur, Samantha Claudet, Pavel Gajdusek, Can Gören, Qilong Gu, Anna Kaminska, Tomasz Kaminski, Ruby Kuo, Akiko Kyuba, Jongho Lee, Kartik Mathur, Petter Merok, Ivana Milovanović, Nani Paananen, Vesa-Matti Paananen, Anna Pavlenko, Bruno Pereira Vidal, Luciano Strika, Yueh Tsao, Davide Turcato, Oleksandr Vakhno, Judit Velcsov, Anna Vickers, Stéphanie Visser, Herdyan Widarmanto, Andrey Zaikin, Si-Qing Chen

机构 * Microsoft Research India（微软印度研究院）

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.CY、cs.LG

Comments AAAI 2025--camera ready + extended abstract

2504.19066 2025-04-29 cs.CL cs.AI cs.LG physics.ao-ph 67%

ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics

Deeksha Varshney, Keane Ong, Rui Mao, Erik Cambria, Gianmarco Mengaldo

机构 * College of Design and Engineering, National University of Singapore（新加坡国立大学设计与工程学院）； Department of Mathematics, National University of Singapore（新加坡国立大学数学系）； College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

2504.12545 2025-04-18 cs.CY cs.AI cs.CL 67%

Knowledge Acquisition on Mass-shooting Events via LLMs for AI-Driven Justice

Benign John Ihugba, Afsana Nasrin, Ling Wu, Lin Li, Lijun Qian, Xishuang Dong

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

2504.11426 2025-04-16 cs.CL cs.AI cs.LG 67%

A Dual-Space Framework for General Knowledge Distillation of Large Language Models

Xue Zhang, Songming Zhang, Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 9 figures, 11 tables, under review. Code is available at: https://github.com/songmzhang/DSKDv2. arXiv admin note: text overlap with arXiv:2406.17328

2502.14202 2025-04-07 cs.SE cs.AI cs.CL cs.LG 67%

Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions

Amirali Sajadi, Binh Le, Anh Nguyen, Kostadin Damevski, Preetha Chatterjee

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMSE

2410.07627 2025-03-21 cs.LG cs.AI cs.CL stat.ML 67%

Automatic Curriculum Expert Iteration for Reliable LLM Reasoning

Zirui Zhao, Hanze Dong, Amrita Saha, Caiming Xiong, Doyen Sahoo

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages

2503.06074 2025-03-11 cs.CL cs.AI cs.LG 67%

Towards Conversational AI for Disease Management

Anil Palepu, Valentin Liévin, Wei-Hung Weng, Khaled Saab, David Stutz, Yong Cheng, Kavita Kulkarni, S. Sara Mahdavi, Joëlle Barral, Dale R. Webster, Katherine Chou, Avinatan Hassidim, Yossi Matias, James Manyika, Ryutaro Tanno, Vivek Natarajan, Adam Rodman, Tao Tu, Alan Karthikesalingam, Mike Schaekermann

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 62 pages, 7 figures in main text, 36 figures in appendix

2407.00617 2025-03-04 cs.LG cs.AI cs.CL cs.GT 67%

Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning

Yuheng Zhang, Dian Yu, Baolin Peng, Linfeng Song, Ye Tian, Mingyue Huo, Nan Jiang, Haitao Mi, Dong Yu

专题命中安全训练：RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

1. 安全训练 3266 篇

Collapse of Irrelevant Representations (CIR) Ensures Robust and Non-Disruptive LLM Unlearning

Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI

Building a Foundational Guardrail for General Agentic Systems via Synthetic Data

AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal Imitation-Exploration Balance

Reinforcement Learning for Clinical Reasoning: Aligning LLMs with ACR Imaging Appropriateness Criteria

Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains

Philosophy-informed Machine Learning

Automating Steering for Safe Multimodal Large Language Models

LogicGuard: Improving Embodied LLM agents through Temporal Logic based Critics

Iterative Prompt Refinement for Safer Text-to-Image Generation

The Model Hears You: Audio Language Model Deployments Should Consider the Principle of Least Privilege

Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation

AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air

Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy

Multimodal Framework for Explainable Autonomous Driving: Integrating Video, Sensor, and Textual Data for Enhanced Decision-Making and Transparency

Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool

7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement

On the Surprising Efficacy of LLMs for Penetration-Testing

NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generation

Learning to Route LLMs with Confidence Tokens

ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails

PPO-BR: Dual-Signal Entropy-Reward Adaptation for Trust Region Policy Optimization

RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?

ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics

Knowledge Acquisition on Mass-shooting Events via LLMs for AI-Driven Justice

A Dual-Space Framework for General Knowledge Distillation of Large Language Models

Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions

Automatic Curriculum Expert Iteration for Reliable LLM Reasoning

Towards Conversational AI for Disease Management

Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning