大模型对齐与安全 - arXivDaily 专题

2405.16833 2025-01-07 cs.LG 79%

Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models

Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yu Chen, Chia-Mu Yu, Chun-Ying Huang

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments This is the camera-ready version accepted for NeurIPS 2024

2412.06878 2024-12-11 cs.CV cs.LG 79%

SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations

Zhaorun Chen, Francesco Pinto, Minzhou Pan, Bo Li

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments 43 pages, 20 figures

2412.03253 2024-12-05 cs.CL 79%

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs

Juhao Liang, Zhenyang Cai, Jianqing Zhu, Huang Huang, Kewei Zong, Bang An, Mosen Alharthi, Juncai He, Lian Zhang, Haizhou Li, Benyou Wang, Jinchao Xu

专题命中安全训练：alignment(title,abstract)；分类 cs.CL

Comments Accepted to NeurIPS 2024 main conference. see https://github.com/FreedomIntelligence/AceGPT-v2

2411.12183 2024-11-20 eess.SY cs.LG cs.SY 79%

Action-Attentive Deep Reinforcement Learning for Autonomous Alignment of Beamlines

Siyu Wang, Shengran Dai, Jianhui Jiang, Shuang Wu, Yufei Peng, Junbin Zhang

专题命中安全训练：alignment(title,abstract)；分类 cs.LG

Comments 17 pages, 5 figures

2409.11365 2024-10-10 cs.CL 79%

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

Jiahui Gao, Renjie Pi, Tianyang Han, Han Wu, Lanqing Hong, Lingpeng Kong, Xin Jiang, Zhenguo Li

专题命中安全训练：safety(title,abstract)；分类 cs.CL

Comments 10 pages, COLM-2024

2409.07878 2024-09-26 cs.CY 79%

Mapping Technical Safety Research at AI Companies: A literature review and incentives analysis

Oscar Delaney, Oliver Guest, Zoe Williams

专题命中安全训练：safety(title,abstract)；分类 cs.CY

Comments Sep-25 update adds papers that we missed in the original dataset, as well as more acknowledgements

2409.10188 2024-09-17 cs.LG 79%

Enhancing RL Safety with Counterfactual LLM Reasoning

Dennis Gross, Helge Spieker

专题命中安全训练：safety(title,abstract)；分类 cs.LG

2405.08443 2024-09-04 cs.LG 79%

Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control

Yang Qu, Jinming Ma, Feng Wu

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments Accepted by IJCAI2024

2408.04522 2024-08-09 cs.CL 79%

Compromesso! Italian Many-Shot Jailbreaks Undermine the Safety of Large Language Models

Fabio Pernisi, Dirk Hovy, Paul Röttger

专题命中安全训练：safety(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2024 (Student Research Workshop)

2408.00997 2024-08-05 cs.AI 79%

A Safe Exploration Strategy for Model-free Task Adaptation in Safety-constrained Grid Environments

Erfan Entezami, Mahsa Sahebdel, Dhawal Gupta

专题命中安全训练：safety(title,abstract)；分类 cs.AI

2407.20729 2024-07-31 cs.CL 79%

Adapting Safe-for-Work Classifier for Malaysian Language Text: Enhancing Alignment in LLM-Ops Framework

Aisyah Razak, Ariff Nazhan, Kamarul Adha, Wan Adzhar Faiq Adzlan, Mas Aisyah Ahmad, Ammar Azman

专题命中安全训练：alignment(title,abstract)；分类 cs.CL

2310.06903 2024-07-16 cs.RO cs.AI 79%

Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimization

Fan Yang, Wenxuan Zhou, Zuxin Liu, Ding Zhao, David Held

专题命中安全训练：safety(title,abstract)；分类 cs.AI

2407.06851 2024-07-10 cs.CL 79%

Safe-Embed: Unveiling the Safety-Critical Knowledge of Sentence Encoders

Jinseok Kim, Jaewon Jung, Sangyeop Kim, Sohyung Park, Sungzoon Cho

专题命中安全训练：safety(title,abstract)；分类 cs.CL

Comments ACL 2024 KnowledgeableLMs workshop paper

2407.04343 2024-07-08 cs.RO cs.LG 79%

Enhancing Safety for Autonomous Agents in Partly Concealed Urban Traffic Environments Through Representation-Based Shielding

Pierre Haritz, David Wanke, Thomas Liebig

专题命中安全训练：safety(title,abstract)；分类 cs.LG

2402.11889 2024-06-18 cs.CL 79%

ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

专题命中安全训练：safety(title,abstract)；分类 cs.CL

Comments Accepted to ACL2024 Findings

2406.05883 2024-06-11 cs.LG cs.IT math.IT stat.ML 79%

Information Theoretic Guarantees For Policy Alignment In Large Language Models

Youssef Mroueh

专题命中安全训练：alignment(title,abstract)；分类 cs.LG

2212.03228 2024-06-11 cs.LG cs.RO cs.SY eess.SY 79%

ISAACS: Iterative Soft Adversarial Actor-Critic for Safety

Kai-Chieh Hsu, Duy Phuong Nguyen, Jaime Fernández Fisac

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments Accepted in 5th Annual Learning for Dynamics & Control Conference (L4DC), University of Pennsylvania

2406.01931 2024-06-06 cs.CL 79%

Dishonesty in Helpful and Harmless Alignment

Youcheng Huang, Jingkun Tang, Duanyu Feng, Zheng Zhang, Wenqiang Lei, Jiancheng Lv, Anthony G. Cohn

专题命中安全训练：alignment(title,abstract)；分类 cs.CL

2405.19414 2024-05-31 cs.LG 79%

Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning

Alexander Politowicz, Sahisnu Mazumder, Bing Liu

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments 9 pages, 3 figures

2405.14488 2024-05-24 cs.CL 79%

MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usability

Yanrui Du, Sendong Zhao, Danyang Zhao, Ming Ma, Yuhan Chen, Liangyu Huo, Qing Yang, Dongliang Xu, Bing Qin

专题命中安全训练：safety(title,abstract)；分类 cs.CL

2405.11034 2024-05-21 cs.LG 79%

Safety in Graph Machine Learning: Threats and Safeguards

Song Wang, Yushun Dong, Binchi Zhang, Zihan Chen, Xingbo Fu, Yinhan He, Cong Shen, Chuxu Zhang, Nitesh V. Chawla, Jundong Li

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments 20 pages

2312.01855 2024-04-03 cs.RO cs.AI 79%

Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters

Aksel Vaaler, Svein Jostein Husa, Daniel Menges, Thomas Nakken Larsen, Adil Rasheed

专题命中安全训练：safety(title,abstract)；分类 cs.AI

Comments 15 pages, 15 figures

2403.14678 2024-03-25 cs.LG 79%

Towards a Framework for Deep Learning Certification in Safety-Critical Applications Using Inherently Safe Design and Run-Time Error Detection

Romeo Valentin

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments Master Thesis

2403.12114 2024-03-20 cs.SE cs.AI 79%

Safety Analysis of Autonomous Railway Systems: An Introduction to the SACRED Methodology

Josh Hunter, John McDermid, Simon Burton

专题命中安全训练：safety(title,abstract)；分类 cs.AI

Comments S. Bernardi, T. Zoppi (Editors), "Fast Abstracts and Student Forum Proceedings - EDCC 2024 - 19th European Dependable Computing Conference, Leuven, Belgium, 8-11 April 2024"

2402.04182 2024-02-07 cs.LG cs.RO 79%

Reinforcement Learning with Ensemble Model Predictive Safety Certification

Sven Gronauer, Tom Haider, Felippe Schmoeller da Roza, Klaus Diepold

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments Published in: Proc. of the 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2024)

2401.00996 2024-01-03 cs.AI cs.CR cs.SE 79%

Safety and Performance, Why Not Both? Bi-Objective Optimized Model Compression against Heterogeneous Attacks Toward AI Software Deployment

Jie Zhu, Leye Wang, Xiao Han, Anmin Liu, Tao Xie

专题命中安全训练：safety(title,abstract)；分类 cs.AI

Comments Accepted by IEEE Transactions on Software Engineering (TSE). Camera-ready Version. arXiv admin note: substantial text overlap with arXiv:2208.05969

2309.09919 2023-11-29 cs.RO cs.AI cs.FL 79%

Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents

Ziyi Yang, Shreyas S. Raman, Ankit Shah, Stefanie Tellex

专题命中安全训练：safety(title,abstract)；分类 cs.AI

2310.19007 2023-11-01 cs.LG 79%

Behavior Alignment via Reward Function Optimization

Dhawal Gupta, Yash Chandak, Scott M. Jordan, Philip S. Thomas, Bruno Castro da Silva

专题命中安全训练：alignment(title,abstract)；分类 cs.LG

Comments (Spotlight) Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

2309.10657 2023-10-05 cs.RO cs.LG cs.MA cs.SY eess.SY 79%

Learning Adaptive Safety for Multi-Agent Systems

Luigi Berducci, Shuo Yang, Rahul Mangharam, Radu Grosu

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments Update with appendix

2309.08709 2023-09-19 stat.ML cs.LG 79%

Price of Safety in Linear Best Arm Identification

Xuedong Shang, Igor Colin, Merwan Barlier, Hamza Cherkaoui

专题命中安全训练：safety(title,abstract)；分类 cs.LG

Comments 20 pages, 1 figures

1. 安全训练 3248 篇

Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models

SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs

Action-Attentive Deep Reinforcement Learning for Autonomous Alignment of Beamlines

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

Mapping Technical Safety Research at AI Companies: A literature review and incentives analysis

Enhancing RL Safety with Counterfactual LLM Reasoning

Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control

Compromesso! Italian Many-Shot Jailbreaks Undermine the Safety of Large Language Models

A Safe Exploration Strategy for Model-free Task Adaptation in Safety-constrained Grid Environments

Adapting Safe-for-Work Classifier for Malaysian Language Text: Enhancing Alignment in LLM-Ops Framework

Reinforcement Learning in a Safety-Embedded MDP with Trajectory Optimization

Safe-Embed: Unveiling the Safety-Critical Knowledge of Sentence Encoders

Enhancing Safety for Autonomous Agents in Partly Concealed Urban Traffic Environments Through Representation-Based Shielding

ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding

Information Theoretic Guarantees For Policy Alignment In Large Language Models

ISAACS: Iterative Soft Adversarial Actor-Critic for Safety

Dishonesty in Helpful and Harmless Alignment

Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning

MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usability

Safety in Graph Machine Learning: Threats and Safeguards

Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters

Towards a Framework for Deep Learning Certification in Safety-Critical Applications Using Inherently Safe Design and Run-Time Error Detection

Safety Analysis of Autonomous Railway Systems: An Introduction to the SACRED Methodology

Reinforcement Learning with Ensemble Model Predictive Safety Certification

Safety and Performance, Why Not Both? Bi-Objective Optimized Model Compression against Heterogeneous Attacks Toward AI Software Deployment

Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents

Behavior Alignment via Reward Function Optimization

Learning Adaptive Safety for Multi-Agent Systems

Price of Safety in Linear Best Arm Identification