大模型对齐与安全 - arXivDaily 专题

2407.01903 2024-11-01 cs.LG cs.AI cs.CV 62%

Text-Aware Diffusion for Policy Learning

Calvin Luo, Mandy He, Zilai Zeng, Chen Sun

专题命中安全训练：alignment(abstract)；分类 cs.AI、cs.LG

2405.14577 2024-11-01 cs.CL cs.LG 62%

Representation Noising: A Defence Mechanism Against Harmful Finetuning

Domenic Rosati, Jan Wehner, Kai Williams, Łukasz Bartoszcze, David Atanasov, Robie Gonzales, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, Frank Rudzicz

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.LG

Comments Published in NeurIPs 2024

2204.10438 2024-10-28 cs.AI cs.LG cs.NE 62%

EVOTER: Evolution of Transparent Explainable Rule-sets

Hormoz Shahrzad, Babak Hodjat, Risto Miikkulainen

专题命中安全训练：trustworthy(abstract)；分类 cs.AI、cs.LG

2410.12937 2024-10-18 cs.CL cs.LG 62%

Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging

Jacob Morrison, Noah A. Smith, Hannaneh Hajishirzi, Pang Wei Koh, Jesse Dodge, Pradeep Dasigi

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.LG

Comments Findings of EMNLP 2024

2410.10871 2024-10-16 cs.CL cs.AI 62%

Applying Refusal-Vector Ablation to Llama 3.1 70B Agents

Simon Lermen, Mateusz Dziemian, Govind Pimpale

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

2410.06491 2024-10-10 cs.AI cs.LG 62%

Honesty to Subterfuge: In-Context Reinforcement Learning Can Make Honest Models Reward Hack

Leo McKee-Reid, Christoph Sträter, Maria Angelica Martinez, Joe Needham, Mikita Balesni

专题命中安全训练：alignment(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 9 figures

2410.06384 2024-10-10 cs.AI cs.CL cs.IR 62%

Validation of the Scientific Literature via Chemputation Augmented by Large Language Models

Sebastian Pagel, Michael Jirasek, Leroy Cronin

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 7 figures, 34 references

2410.02657 2024-10-04 cs.CL cs.CY 62%

Hate Personified: Investigating the role of LLMs in content moderation

Sarah Masud, Sahajpreet Singh, Viktor Hangya, Alexander Fraser, Tanmoy Chakraborty

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.CY

Comments 17 pages, 6 Figures, 13 Tables, EMNLP'24 Mains

2409.19038 2024-10-02 cs.AI cs.LG cs.MA cs.RO 62%

Intention-aware policy graphs: answering what, how, and why in opaque agents

Victor Gimenez-Abalos, Sergio Alvarez-Napagao, Adrian Tormos, Ulises Cortés, Javier Vázquez-Salceda

专题命中安全训练：trustworthy(abstract)；分类 cs.AI、cs.LG

Comments 57 pages, 8 figures, 5 tables

2409.20517 2024-10-01 cs.LG cs.AI 62%

SMLE: Safe Machine Learning via Embedded Overapproximation

Matteo Francobaldi, Michele Lombardi

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2409.15461 2024-09-25 cs.AI cs.CL 62%

RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration

Haoyu Huang, Tong Niu, Rui Yang, Luping Shi

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

2305.13681 2024-09-25 cs.LG cs.AI cs.RO 62%

GUARD: A Safe Reinforcement Learning Benchmark

Weiye Zhao, Yifan Sun, Feihan Li, Rui Chen, Ruixuan Liu, Tianhao Wei, Changliu Liu

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments Published in Transaction of Machine Learning Research

2309.00236 2024-09-19 cs.LG cs.CL cs.CR 62%

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.LG

Comments Project page at https://image-hijacks.github.io

2403.18209 2024-09-13 cs.LG cs.AI cs.RO 62%

Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving

Xuemin Hu, Pan Chen, Yijun Wen, Bo Tang, Long Chen

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2312.03814 2024-09-04 cs.LG cs.AI 62%

Pearl: A Production-ready Reinforcement Learning Agent

Zheqing Zhu, Rodrigo de Salvo Braz, Jalaj Bhandari, Daniel Jiang, Yi Wan, Yonathan Efroni, Liyuan Wang, Ruiyang Xu, Hongbo Guo, Alex Nikulkov, Dmytro Korenkevych, Urun Dogan, Frank Cheng, Zheng Wu, Wanqiao Xu

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Journal ref Journal of Machine Learning Research, 2024

2403.15826 2024-08-29 eess.SY cs.AI cs.LG cs.RO cs.SY 62%

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

Navid Hashemi, Bardh Hoxha, Danil Prokhorov, Georgios Fainekos, Jyotirmoy Deshmukh

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2402.04580 2024-08-28 cs.RO cs.AI cs.LG 62%

A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents

Haoyi Niu, Jianming Hu, Guyue Zhou, Xianyuan Zhan

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments IJCAI 2024

2408.11021 2024-08-21 cs.CL cs.AI cs.MA 62%

Athena: Safe Autonomous Agents with Verbal Contrastive Learning

Tanmana Sadhu, Ali Pesaranghader, Yanan Chen, Dong Hoon Yi

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 2 figures, 4 tables

2408.07962 2024-08-16 cs.LG cs.AI cs.RO cs.SY eess.SY 62%

Meta SAC-Lag: Towards Deployable Safe Reinforcement Learning via MetaGradient-based Hyperparameter Tuning

Homayoun Honari, Amir Mehdi Soufi Enayati, Mehran Ghafarian Tamizi, Homayoun Najjaran

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments Main text accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024, 10 pages, 4 figures, 3 tables

2407.19760 2024-08-12 cs.CL cs.CY 62%

Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios

Camilla Bignotti, Carolina Camassa

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.CY

Comments Accepted at AIES24

2402.17649 2024-08-12 cs.CL cs.CY 62%

Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs

Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, Sebastian Padó

专题命中安全训练：alignment(abstract)；分类 cs.CL、cs.CY

Comments 12 pages, TACL

2407.21772 2024-08-06 cs.CL cs.LG 62%

ShieldGemma: Generative AI Content Moderation Based on Gemma

Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, Oscar Wahltinez

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.LG

2312.08381 2024-07-31 cs.LG cs.AI 62%

An Explainable Machine Learning Framework for the Accurate Diagnosis of Ovarian Cancer

Asif Newaz, Abdullah Taharat, Md Sakibul Islam, A. G. M. Fuad Hasan Akanda

专题命中安全训练：trustworthy(abstract)；分类 cs.AI、cs.LG

2407.15718 2024-07-23 cs.CY cs.AI cs.HC cs.IR cs.SE 62%

Integrating AI Tutors in a Programming Course

Iris Ma, Alberto Krone Martins, Cristina Videira Lopes

专题命中安全训练：alignment(abstract)；分类 cs.AI、cs.CY

Comments Accepted at SIGCSE Virtual 2024

2407.10995 2024-07-22 cs.CL cs.AI 62%

LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content

Jessica Foo, Shaun Khoo

专题命中安全训练：safety(abstract)；分类 cs.CL、cs.AI

Comments Preprint

2205.06212 2024-07-17 eess.SY cs.AI cs.LG cs.SY 62%

Contingency-constrained economic dispatch with safe reinforcement learning

Michael Eichelbeck, Hannah Markgraf, Matthias Althoff

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Journal ref 21st IEEE International Conference on Machine Learning and Applications (ICMLA), 2022, pp. 597-602

2407.05580 2024-07-09 cs.LG cs.AI 62%

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model

Zepeng Wang, Chao Ma, Linjiang Zhou, Libing Wu, Lei Yang, Xiaochuan Shi, Guojun Peng

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2402.09132 2024-07-09 cs.AI cs.LG 62%

Exploring the Adversarial Capabilities of Large Language Models

Lukas Struppek, Minh Hieu Le, Dominik Hintersdorf, Kristian Kersting

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2301.11461 2024-07-08 cs.LG cs.AI 62%

Learning to Generate All Feasible Actions

Mirco Theile, Daniele Bernardini, Raphael Trumpp, Cristina Piazza, Marco Caccamo, Alberto L. Sangiovanni-Vincentelli

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

2407.00004 2024-07-02 q-bio.BM cs.AI cs.LG q-bio.QM 62%

Multi-objective generative AI for designing novel brain-targeting small molecules

Ayush Noori, Iñaki Arango, William E. Byrd, Nada Amin

专题命中安全训练：safety(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 4 figures, Generative and Experimental Perspectives for Biomolecular Design Workshop at the 12th International Conference on Learning Representations

1. 安全训练 3278 篇

Text-Aware Diffusion for Policy Learning

Representation Noising: A Defence Mechanism Against Harmful Finetuning

EVOTER: Evolution of Transparent Explainable Rule-sets

Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging

Applying Refusal-Vector Ablation to Llama 3.1 70B Agents

Honesty to Subterfuge: In-Context Reinforcement Learning Can Make Honest Models Reward Hack

Validation of the Scientific Literature via Chemputation Augmented by Large Language Models

Hate Personified: Investigating the role of LLMs in content moderation

Intention-aware policy graphs: answering what, how, and why in opaque agents

SMLE: Safe Machine Learning via Embedded Overapproximation

RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration

GUARD: A Safe Reinforcement Learning Benchmark

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving

Pearl: A Production-ready Reinforcement Learning Agent

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents

Athena: Safe Autonomous Agents with Verbal Contrastive Learning

Meta SAC-Lag: Towards Deployable Safe Reinforcement Learning via MetaGradient-based Hyperparameter Tuning

Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios

Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs

ShieldGemma: Generative AI Content Moderation Based on Gemma

An Explainable Machine Learning Framework for the Accurate Diagnosis of Ovarian Cancer

Integrating AI Tutors in a Programming Course

LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content

Contingency-constrained economic dispatch with safe reinforcement learning

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model

Exploring the Adversarial Capabilities of Large Language Models

Learning to Generate All Feasible Actions

Multi-objective generative AI for designing novel brain-targeting small molecules