大模型对齐与安全 - arXivDaily 专题

2410.24190 2024-11-12 cs.CL cs.CY 62%

Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters

Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, Dawn Song

专题命中偏好对齐：safety(abstract)；分类 cs.CL、cs.CY

Comments EMNLP 2024 Main

2411.05232 2024-11-11 cs.CL cs.AI 62%

Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities

Shengzhi Li, Kittipat Kampa, Rongyu Lin, Bohang Li, Shichao Pei

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.AI

Comments We share our latest dataset on https://github.com/findalexli/Abstract2Appendix

2405.16964 2024-11-11 cs.CL cs.AI 62%

Exploring the LLM Journey from Cognition to Expression with Linear Representations

Yuzi Yan, Jialian Li, Yipin Zhang, Dong Yan

专题命中偏好对齐：RLHF(abstract)；分类 cs.CL、cs.AI

Comments Published in ICML 2024

2405.17382 2024-11-08 cs.LG cs.CL 62%

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

Hyunseok Lee, Jihoon Tack, Jinwoo Shin

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.LG

Comments Published as a conference proceeding for NeurIPS 2024

2410.18634 2024-11-05 cs.CL cs.AI cs.IR 62%

Little Giants: Synthesizing High-Quality Embedding Data at Scale

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.AI

2405.14622 2024-11-05 cs.LG cs.CL cs.CV 62%

Calibrated Self-Rewarding Vision Language Models

Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, Huaxiu Yao

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.LG

Comments Added some experiments and charts, and redrew some figures V4

2407.00072 2024-11-04 cs.IR cs.AI cs.CL 62%

Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback

Yu Bai, Yukai Miao, Li Chen, Dawei Wang, Dan Li, Yanyu Ren, Hongtao Xie, Ce Yang, Xuhui Cai

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.AI

2410.20305 2024-11-01 cs.LG cs.CL 62%

Accelerating Direct Preference Optimization with Prefix Sharing

Franklin Wang, Sumanth Hegde

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.LG

Comments To appear in NeurIPS 2024 in the Fine-Tuning in Machine Learning Workshop

2408.13296 2024-10-31 cs.LG cs.CL 62%

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities

Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, Arsalan Shahid

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.LG

2405.15230 2024-10-30 cs.AI cs.LG 62%

$i$REPO: $i$mplicit Reward Pairwise Difference based Empirical Preference Optimization

Long Tan Le, Han Shu, Tung-Anh Nguyen, Choong Seon Hong, Nguyen H. Tran

专题命中偏好对齐：alignment(abstract)；分类 cs.AI、cs.LG

Comments Under Review

2410.21252 2024-10-29 cs.CL cs.LG 62%

LongReward: Improving Long-context Large Language Models with AI Feedback

Jiajie Zhang, Zhongni Hou, Xin Lv, Shulin Cao, Zhenyu Hou, Yilin Niu, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.LG

2406.10216 2024-10-24 cs.CL cs.AI 62%

Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs

Rui Yang, Ruomeng Ding, Yong Lin, Huan Zhang, Tong Zhang

专题命中偏好对齐：RLHF(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

2410.14807 2024-10-22 cs.LG cs.AI 62%

Aligning AI Agents via Information-Directed Sampling

Hong Jun Jeon, Benjamin Van Roy

专题命中偏好对齐：alignment(abstract)；分类 cs.AI、cs.LG

2309.02915 2024-10-22 cs.CL cs.LG 62%

Persona-aware Generative Model for Code-mixed Language

Ayan Sengupta, Md Shad Akhtar, Tanmoy Chakraborty

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.LG

Comments Published in Transactions on Machine Learning Research (TMLR)

2410.14001 2024-10-21 cs.LG cs.CL 62%

Personalized Adaptation via In-Context Preference Learning

Allison Lau, Younwoo Choi, Vahid Balazadeh, Keertana Chidambaram, Vasilis Syrgkanis, Rahul G. Krishnan

专题命中偏好对齐：RLHF(abstract)；分类 cs.CL、cs.LG

2406.17969 2024-10-17 cs.CL cs.AI 62%

Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective

Hanqi Yan, Yanzheng Xiang, Guangyi Chen, Yifei Wang, Lin Gui, Yulan He

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.AI

Comments EMNLP24, Main, Long

2402.10052 2024-10-17 cs.CL cs.AI 62%

UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models

Yijiang River Dong, Hongzhou Lin, Mikhail Belkin, Ramon Huerta, Ivan Vulić

专题命中偏好对齐：safety(abstract)；分类 cs.CL、cs.AI

2410.11055 2024-10-16 cs.CL cs.AI 62%

Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only

Jihan Yao, Wenxuan Ding, Shangbin Feng, Lucy Lu Wang, Yulia Tsvetkov

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.AI

2402.00658 2024-10-16 cs.AI cs.CL 62%

Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing

Fangkai Jiao, Chengwei Qin, Zhengyuan Liu, Nancy F. Chen, Shafiq Joty

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 9 figures. EMNLP 2024

2410.10801 2024-10-15 cs.CL cs.LG 62%

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning

Aakanksha, Arash Ahmadian, Seraphina Goldfarb-Tarrant, Beyza Ermis, Marzieh Fadaee, Sara Hooker

专题命中偏好对齐：safety(abstract)；分类 cs.CL、cs.LG

2410.06961 2024-10-10 cs.CL cs.AI 62%

Self-Boosting Large Language Models with Synthetic Preference Data

Qingxiu Dong, Li Dong, Xingxing Zhang, Zhifang Sui, Furu Wei

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.AI

2410.06508 2024-10-10 cs.LG cs.CL 62%

Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning

Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu, Baolin Peng, Haitao Mi, Furong Huang, Dong Yu

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.LG

2405.00747 2024-10-07 cs.LG cs.AI 62%

Soft Preference Optimization: Aligning Language Models to Expert Distributions

Arsalan Sharifnassab, Saber Salehkaleybar, Sina Ghiassian, Surya Kanoria, Dale Schuurmans

专题命中偏好对齐：alignment(abstract)；分类 cs.AI、cs.LG

2406.08116 2024-10-07 cs.CL cs.AI 62%

Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling

Zile Qiao, Wei Ye, Yong Jiang, Tong Mo, Pengjun Xie, Weiping Li, Fei Huang, Shikun Zhang

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.AI

2409.18618 2024-10-03 cs.CL cs.AI 62%

Model-based Preference Optimization in Abstractive Summarization without Human Feedback

Jaepill Choi, Kyubyung Chae, Jiwoo Song, Yohan Jo, Taesup Kim

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024

2409.08642 2024-10-02 cs.AI cs.LG 62%

CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks

Tianlong Wang, Junzhe Chen, Xueting Han, Jing Bai

专题命中偏好对齐：DPO(abstract)；分类 cs.AI、cs.LG

2409.03444 2024-09-06 cs.CL cond-mat.mtrl-sci cs.AI 62%

Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities

Wei Lu, Rachel K. Luu, Markus J. Buehler

专题命中偏好对齐：DPO(abstract)；分类 cs.CL、cs.AI

2409.01552 2024-09-04 cs.CL cs.AI 62%

Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

Zhuo Li, Yuhao Du, Jinpeng Hu, Xiang Wan, Anningzhe Gao

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.AI

2408.16753 2024-08-30 cs.CL cs.LG 62%

Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models

Alec Solway

专题命中偏好对齐：alignment(abstract)；分类 cs.CL、cs.LG

2408.16032 2024-08-30 cs.LG cs.AI cs.IR 62%

An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenders

Shuang Feng, Grace Feng

专题命中偏好对齐：DPO(abstract)；分类 cs.AI、cs.LG

1. 偏好对齐 3243 篇

Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters

Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities

Exploring the LLM Journey from Cognition to Expression with Linear Representations

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

Little Giants: Synthesizing High-Quality Embedding Data at Scale

Calibrated Self-Rewarding Vision Language Models

Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback

Accelerating Direct Preference Optimization with Prefix Sharing

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities

$i$REPO: $i$mplicit Reward Pairwise Difference based Empirical Preference Optimization

LongReward: Improving Long-context Large Language Models with AI Feedback

Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs

Aligning AI Agents via Information-Directed Sampling

Persona-aware Generative Model for Code-mixed Language

Personalized Adaptation via In-Context Preference Learning

Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective

UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models

Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only

Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning

Self-Boosting Large Language Models with Synthetic Preference Data

Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning

Soft Preference Optimization: Aligning Language Models to Expert Distributions

Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling

Model-based Preference Optimization in Abstractive Summarization without Human Feedback

CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks

Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities

Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models

An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenders