大模型对齐与安全 - arXivDaily 专题

2412.06869 2024-12-11 cs.LG cs.AI cs.CV cs.SE 81%

Safety Monitoring of Machine Learning Perception Functions: a Survey

Raul Sena Ferreira, Joris Guérin, Kevin Delmas, Jérémie Guiochet, Hélène Waeselynck

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 2 figures

2407.20208 2024-12-02 cs.AI cs.LG cs.NE 81%

Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence

James M. Mazzu

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

2407.19860 2024-11-06 cs.LG cs.AI 81%

Anomalous State Sequence Modeling to Enhance Safety in Reinforcement Learning

Leen Kweider, Maissa Abou Kassem, Ubai Sandouk

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Journal ref IEEE Access, vol. 12, pp. 157140-157148, Oct. 25, 2024

2406.10847 2024-10-29 cs.AI cs.CE cs.CL cs.MA 81%

TorchOpera: A Compound AI System for LLM Safety

Shanshan Han, Zijian Hu, Alay Dilipbhai Shah, Han Jin, Yuhang Yao, Dimitris Stripelis, Zhaozhuo Xu, Chaoyang He

专题命中安全训练：safety(title,abstract)；分类 cs.CL、cs.AI

2410.15489 2024-10-22 cs.RO cs.AI cs.LG 81%

Generative AI Agents in Autonomous Machines: A Safety Perspective

Jason Jabbour, Vijay Janapa Reddi

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2410.10014 2024-10-15 cs.CL cs.AI 81%

Safety-Aware Fine-Tuning of Large Language Models

Hyeong Kyu Choi, Xuefeng Du, Yixuan Li

专题命中安全训练：safety(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Workshop on Safe Generative AI

2310.12567 2024-10-08 cs.AI cs.LG 81%

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Jiaming Ji, Borong Zhang, Jiayi Zhou, Xuehai Pan, Weidong Huang, Ruiyang Sun, Yiran Geng, Yifan Zhong, Juntao Dai, Yaodong Yang

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Published at NeurIPS 2023

2206.09743 2024-09-13 cs.AI cs.LG cs.SY eess.SY 81%

Guided Safe Shooting: model based reinforcement learning with safety constraints

Giuseppe Paolo, Jonas Gonzalez-Billandon, Albert Thomas, Balázs Kégl

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2401.00391 2024-08-08 cs.RO cs.AI cs.CV cs.LG 81%

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries

Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by ECCV2024; Project website: https://safe-sim.github.io/

2403.10462 2024-03-20 cs.CY cs.AI 81%

Safety Cases: How to Justify the Safety of Advanced AI Systems

Joshua Clymer, Nick Gabrieli, David Krueger, Thomas Larsen

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.CY

2306.17279 2024-03-14 cs.LG cs.AI 81%

Probabilistic Constraint for Safety-Critical Reinforcement Learning

Weiqin Chen, Dharmashankar Subramanian, Santiago Paternain

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments arXiv admin note: text overlap with arXiv:2210.00596

2312.10230 2023-12-19 cs.AI cs.LG 81%

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

Minjae Cho, Chuangchuang Sun

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2309.15025 2023-09-27 cs.CL cs.AI 81%

Large Language Model Alignment: A Survey

Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, Deyi Xiong

专题命中安全训练：alignment(title,abstract)；分类 cs.CL、cs.AI

Comments 76 pages

2307.02867 2023-07-07 cs.SE cs.AI cs.LG cs.SY eess.SY 81%

Towards a safe MLOps Process for the Continuous Development and Safety Assurance of ML-based Systems in the Railway Domain

Marc Zeller, Thomas Waschulzik, Reiner Schmid, Claus Bahlmann

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2306.06135 2023-06-13 cs.LG cs.AI 81%

Safety and Fairness for Content Moderation in Generative Models

Susan Hao, Piyush Kumar, Sarah Laszlo, Shivani Poddar, Bhaktipriya Radharapu, Renee Shelby

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments CVPR Workshop Paper

2305.04727 2023-05-09 cs.LG cs.AI 81%

DEFENDER: DTW-Based Episode Filtering Using Demonstrations for Enhancing RL Safety

André Correia, Luís Alexandre

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

2302.09318 2023-02-21 cs.LG cs.AI 81%

Effective Multimodal Reinforcement Learning with Modality Alignment and Importance Enhancement

Jinming Ma, Feng Wu, Yingfeng Chen, Xianpeng Ji, Yu Ding

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

Comments 10 pages, 12 figures, This article is an extended version of the Extended Abstract accepted by the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-2022)

2301.11857 2023-02-07 cs.AI cs.LG cs.MA 81%

Policy-Value Alignment and Robustness in Search-based Multi-Agent Learning

Niko A. Grupen, Michael Hanlon, Alexis Hao, Daniel D. Lee, Bart Selman

专题命中安全训练：alignment(title,abstract)；分类 cs.AI、cs.LG

Comments 9 pages, 5 figures

2211.10237 2022-11-21 cs.RO cs.AI cs.LG 81%

Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulator

Ho Suk, Taewoo Kim, Hyungbin Park, Pamul Yadav, Junyong Lee, Shiho Kim

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 9 pages including appendices, 4 figures, NeurIPS 2022 Workshop: Machine Learning for Autonomous Driving (ML4AD)

2211.04530 2022-11-10 cs.LG cs.CY 81%

Creating a Safety Assurance Case for an ML Satellite-Based Wildfire Detection and Alert System

Richard Hawkins, Chiara Picardi, Lucy Donnell, Murray Ireland

专题命中安全训练：safety(title,abstract)；分类 cs.CY、cs.LG

2201.12427 2022-10-12 cs.LG cs.AI cs.RO 81%

Towards Safe Reinforcement Learning with a Safety Editor Policy

Haonan Yu, Wei Xu, Haichao Zhang

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2022 camera-ready version

2111.07395 2022-06-24 cs.LG cs.AI cs.RO 81%

Explicit Explore, Exploit, or Escape ($E^4$): near-optimal safety-constrained reinforcement learning in polynomial time

David M. Bossens, Nicholas Bishop

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted at Machine Learning

2201.12819 2022-02-01 cs.AI cs.LG cs.RO cs.SY eess.SY 81%

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

Andrei Aksjonov, Ville Kyrki

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 11 pages, 9 figures, 2 tables

2201.08111 2022-01-25 cs.AI cs.LG 81%

Safety-Aware Multi-Agent Apprenticeship Learning

Junchen Zhao

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 58 pages, 4 figures. Master's report. arXiv admin note: text overlap with arXiv:1710.07983 by other authors

2003.05311 2020-12-08 cs.LG cs.AI 81%

A Safety Framework for Critical Systems Utilising Deep Neural Networks

Xingyu Zhao, Alec Banks, James Sharp, Valentin Robu, David Flynn, Michael Fisher, Xiaowei Huang

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by SafeComp2020

2010.02846 2020-10-07 cs.LG cs.AI 81%

Safety Aware Reinforcement Learning (SARL)

Santiago Miret, Somdeb Majumdar, Carroll Wainwright

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

1909.09142 2019-09-23 cs.LG cs.AI cs.LO stat.ML 81%

Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks

Hao Ren, Sai Krishnan Chandrasekar, Anitha Murugesan

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

1903.02526 2019-04-23 cs.LG cs.AI cs.RO 81%

Safety-Guided Deep Reinforcement Learning via Online Gaussian Process Estimation

Jiameng Fan, Wenchao Li

专题命中安全训练：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 13 pages, 5 figures

2507.16856 2025-10-07 cs.CV cs.AI 80%

SIA: Enhancing Safety via Intent Awareness for Vision-Language Models

Youngjin Na, Sangheon Jeong, Youngwan Lee, Jian Lee, Dawoon Jeong, Youngman Kim

机构 * VLM Safety LAB, MODULABS（视觉语言模型安全实验室，MODULABS）； ETRI（电子技术研究院）； KAIST（韩国科学技术院）

专题命中安全训练：safety(title,abstract)；分类 cs.AI；trustworthy(comments)

Comments Accepted to Safe and Trustworthy Multimodal AI Systems(SafeMM-AI) Workshop at ICCV2025, Non-archival track

2112.08526 2022-05-18 cs.LG cs.RO 80%

Invariance Through Latent Alignment

Takuma Yoneda, Ge Yang, Matthew R. Walter, Bradly Stadie

专题命中安全训练：alignment(title,abstract)；分类 cs.LG

Comments To appear in RSS 2022. Here's our project page: https://invariance-through-latent-alignment.github.io

1. 安全训练 3248 篇

Safety Monitoring of Machine Learning Perception Functions: a Survey

Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence

Anomalous State Sequence Modeling to Enhance Safety in Reinforcement Learning

TorchOpera: A Compound AI System for LLM Safety

Generative AI Agents in Autonomous Machines: A Safety Perspective

Safety-Aware Fine-Tuning of Large Language Models

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Guided Safe Shooting: model based reinforcement learning with safety constraints

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries

Safety Cases: How to Justify the Safety of Advanced AI Systems

Probabilistic Constraint for Safety-Critical Reinforcement Learning

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

Large Language Model Alignment: A Survey

Towards a safe MLOps Process for the Continuous Development and Safety Assurance of ML-based Systems in the Railway Domain

Safety and Fairness for Content Moderation in Generative Models

DEFENDER: DTW-Based Episode Filtering Using Demonstrations for Enhancing RL Safety

Effective Multimodal Reinforcement Learning with Modality Alignment and Importance Enhancement

Policy-Value Alignment and Robustness in Search-based Multi-Agent Learning

Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulator

Creating a Safety Assurance Case for an ML Satellite-Based Wildfire Detection and Alert System

Towards Safe Reinforcement Learning with a Safety Editor Policy

Explicit Explore, Exploit, or Escape ($E^4$): near-optimal safety-constrained reinforcement learning in polynomial time

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

Safety-Aware Multi-Agent Apprenticeship Learning

A Safety Framework for Critical Systems Utilising Deep Neural Networks

Safety Aware Reinforcement Learning (SARL)

Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks

Safety-Guided Deep Reinforcement Learning via Online Gaussian Process Estimation

SIA: Enhancing Safety via Intent Awareness for Vision-Language Models

Invariance Through Latent Alignment