大模型对齐与安全

Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts

Maiya Goloburda, Nurkhan Laiyk, Diana Turmakhan, Yuxia Wang, Mukhammed Togmanov, Jonibek Mansurov, Askhat Sametov, Nurdaulet Mukhituly, Minghan Wang, Daniil Orel, Zain Muhammad Mujahid, Fajri Koto, Timothy Baldwin, Preslav Nakov

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎伊德大学人工智能学院） ； Monash University（莫纳什大学） ； University of Copenhagen（哥本哈根大学） ； The University of Melbourne（墨尔本大学） ； LibrAI（LibrAI公司） ； Institute of Foundation Models（基础模型研究所） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Why Do Some Language Models Fake Alignment While Others Don't?

Abhay Sheshadri, John Hughes, Julian Michael, Alex Mallen, Arun Jose, Janus, Fabien Roger

机构 * Anthropic ； Scale AI ； Redwood Research 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.LG

Generative Grasp Detection and Estimation with Concept Learning-based Safety Criteria

Al-Harith Farhad, Khalil Abuibaid, Christiane Plociennik, Achim Wagner, Martin Ruskowski

机构 * Innovative Factory Solutions（创新工厂解决方案） ； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） ； Chair of Machine Tools and Control Systems (WSKL)（机械工具与控制系统教授职位） ； Rheinland-Pfälzische Technische Universität (RPTU)（莱茵-瓦尔德技术大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments RAAD 2025: 34th International Conference on Robotics in Alpe-Adria-Danube Region

Optimal Transport-Guided Safety in Temporal Difference Reinforcement Learning

Zahra Shahrooei, Ali Baheri

机构 * Department of Mechanical Engineering, Rochester Institute of Technology（机械工程系，罗切斯特理工学院） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

SafeLawBench: Towards Safe Alignment of Large Language Models

Chuxue Cao, Han Zhu, Jiaming Ji, Qichao Sun, Zhenghao Zhu, Yinyu Wu, Juntao Dai, Yaodong Yang, Sirui Han, Yike Guo

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Peking University（北京大学） 纠错

专题命中 安全训练 ：alignment(title)；safety(abstract)；分类 cs.CL

Comments Accepted to ACL2025 Findings

Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learning

Kepu Zhang, Guofu Xie, Weijie Yu, Mingyue Xu, Xu Tang, Yaxin Li, Jun Xu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学耿丽人工智能学院） ； University of International Business and Economics（国际经贸大学） 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.CL

SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMs

Shaona Ghosh, Amrita Bhattacharjee, Yftah Ziser, Christopher Parisien

机构 * NVIDIA（英伟达） ； School of Computing and AI（计算与人工智能学院） ； Arizona State University（亚利桑那州立大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments arXiv admin note: text overlap with arXiv:2410.01174

Should LLM Safety Be More Than Refusing Harmful Instructions?

Utsav Maskey, Mark Dras, Usman Naseem

机构 * Macquarie University（麦考瑞大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments Preprint

Feasibility-Aware Pessimistic Estimation: Toward Long-Horizon Safety in Offline RL

Zhikun Tao

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments arXiv admin comment: This version removed due to inaccurate authorship

Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning

Maeva Guerrier, Karthik Soma, Hassan Fouad, Giovanni Beltrame

机构 * Department of Computer Engineering and Software Engineering, Polytechnique Montreal（计算机工程与软件工程系，蒙特利尔大学） ； Mila - Quebec AI Institute（魁北克人工智能研究所） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Towards Reliable and Interpretable Traffic Crash Pattern Prediction and Safety Interventions Using Customized Large Language Models

Yang Zhao, Pu Wang, Yibo Zhao, Hongru Du, Hao Frank Yang

机构 * Center for Systems Science and Engineering（系统科学与工程中心） ； Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments Last revised 13 Feb 2025. Under review in Nature portfolio

Safety and optimality in learning-based control at low computational cost

Dominik Baumann, Krzysztof Kowalczyk, Cristian R. Rojas, Koen Tiels, Pawel Wachel

机构 * Aalto University（阿alto大学） ； Uppsala University（乌普萨拉大学） ； Wrocław University of Science and Technology（沃斯克拉大学科学与技术大学） ； KTH Royal Institute of Technology（皇家理工学院） ； Eindhoven University of Technology（埃因霍温理工大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.LG

Comments Accepted final version to appear in the IEEE Transactions on Automatic Control

Demonstrating ViSafe: Vision-enabled Safety for High-speed Detect and Avoid

Parv Kapoor, Ian Higgins, Nikhil Keetha, Jay Patrikar, Brady Moon, Zelin Ye, Yao He, Ivan Cisneros, Yaoyu Hu, Changliu Liu, Eunsuk Kang, Sebastian Scherer

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments 13 pages, RSS 2025 Demo track, https://theairlab.org/visafe/

Safe and Efficient CAV Lane Changing using Decentralised Safety Shields

Bharathkumar Hegde, Melanie Bouroche

机构 * School of Computer Science and Statistics, Trinity College Dublin（计算机科学与统计学学院，都柏林三一学院） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Accepted in IEEE IV 2025

Planning Safety Trajectories with Dual-Phase, Physics-Informed, and Transportation Knowledge-Driven Large Language Models

Rui Gan, Pei Li, Keke Long, Bocheng An, Junwei You, Keshu Wu, Bin Ran

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Epistemic Closure and the Irreversibility of Misalignment: Modeling Systemic Barriers to Alignment Innovation

Andy Williams

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI

Graphormer-Guided Task Planning: Beyond Static Rules with LLM Safety Perception

Wanjing Huang, Tongjie Pan, Yalan Ye

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstruction

Jordan Vice, Naveed Akhtar, Mubarak Shah, Richard Hartley, Ajmal Mian

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments This research is supported by the NISDRG project #20100007, funded by the Australian Government

SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal

Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, Prateek Mittal

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Paper accepted to ICLR 2025

Safeguarding AI Agents: Developing and Analyzing Safety Architectures

Ishaan Domkundwar, Mukunda N S, Ishaan Bhola, Riddhik Kochhar

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models

Zihao Sheng, Zilin Huang, Yansong Qu, Yue Leng, Sruthi Bhavanam, Sikai Chen

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection

Weidi Luo, Shenghong Dai, Xiaogeng Liu, Suman Banerjee, Huan Sun, Muhao Chen, Chaowei Xiao

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

How Safe is Your Safety Metric? Automatic Concatenation Tests for Metric Reliability

Ora Nova Fandina, Leshem Choshen, Eitan Farchi, George Kour, Yotam Perlitz, Orna Raz

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Gen-AI for User Safety: A Survey

Akshar Prabhu Desai, Tejasvi Ravi, Mohammad Luqman, Mohit Sharma, Nithya Kota, Pranjul Yadav

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Journal ref 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 5315-5324

Why human-AI relationships need socioaffective alignment

Hannah Rose Kirk, Iason Gabriel, Chris Summerfield, Bertie Vidgen, Scott A. Hale

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI

A sketch of an AI control safety case

Tomek Korbak, Joshua Clymer, Benjamin Hilton, Buck Shlegeris, Geoffrey Irving

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

ASTRAL: Automated Safety Testing of Large Language Models

Miriam Ugarte, Pablo Valle, José Antonio Parejo, Sergio Segura, Aitor Arrieta

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Journal ref The 6th ACM/IEEE International Conference on Automation of Software Test (AST 2025)

MSTS: A Multimodal Safety Test Suite for Vision-Language Models

Paul Röttger, Giuseppe Attanasio, Felix Friedrich, Janis Goldzycher, Alicia Parrish, Rishabh Bhardwaj, Chiara Di Bonaventura, Roman Eng, Gaia El Khoury Geagea, Sujata Goswami, Jieun Han, Dirk Hovy, Seogyeong Jeong, Paloma Jeretič, Flor Miriam Plaza-del-Arco, Donya Rooein, Patrick Schramowski, Anastassia Shaitarova, Xudong Shen, Richard Willats, Andrea Zugarini, Bertie Vidgen

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments under review

Safety through feedback in Constrained RL

Shashank Reddy Chirra, Pradeep Varakantham, Praveen Paruchuri

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI

Comments Accepted at NeurIPS 2024 (Poster)

Autonomous Alignment with Human Value on Altruism through Considerate Self-imagination and Theory of Mind

Haibo Tong, Enmeng Lu, Yinqian Sun, Zhengqiang Han, Chao Liu, Feifei Zhao, Yi Zeng

专题命中 安全训练 ：alignment(title)；safety(abstract)；分类 cs.AI