RAG / 检索增强生成

Code Summarization Beyond Function Level

Vladimir Makharev, Vladimir Ivanov

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted to LLM4Code @ ICSE'25; 8 pages, 3 figures, 4 tables

What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis

Peiran Wang, Yang Liu, Yunfei Lu, Jue Hong, Ye Wu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

LLMs to Support a Domain Specific Knowledge Assistant

Maria-Flavia Lovin

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation

Ziyao Zhang, Yanlin Wang, Chong Wang, Jiachi Chen, Zibin Zheng

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ISSTA 2025

Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, Bo Zheng, Kaifu Zhang

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Citekit: A Modular Toolkit for Large Language Model Citation Generation

Jiajun Shen, Tong Zhou, Yubo Chen, Kang Liu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 7 pages, 14 figures

Attention with Dependency Parsing Augmentation for Fine-Grained Attribution

Qiang Ding, Lvzhou Luo, Yixuan Cao, Ping Luo

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 7 figures, submitted to ACL ARR 2024 October

A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look

Shivani Upadhyay, Ronak Pradeep, Nandan Thakur, Daniel Campos, Nick Craswell, Ian Soboroff, Hoa Trang Dang, Jimmy Lin

专题命中 RAG评测 ：RAG(abstract)；分类 cs.IR、cs.CL

MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents

Liyan Tang, Philippe Laban, Greg Durrett

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

KodeXv0.1: A Family of State-of-the-Art Financial Large Language Models

Neel Rajani, Lilli Kiessling, Aleksandr Ogaltsov, Claus Lang

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 8 figures

Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course

Sebastian Kahl, Felix Löffler, Martin Maciol, Fabian Ridder, Marius Schmitz, Jennifer Spanagel, Jens Wienkamp, Christopher Burgahn, Malte Schilling

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments The article is an extended version of a paper presented at the International Workshop on AI in Education and Educational Research (AIEER) at ECAI-2024 (27th European Conference on Artificial Intelligence)

A Comparison of LLM Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case

Sonia Meyer, Shreya Singh, Bertha Tam, Christopher Ton, Angel Ren

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Multi-step Inference over Unstructured Data

Aditya Kalyanpur, Kailash Karthik Saravanakumar, Victor Barres, CJ McFate, Lori Moon, Nati Seifu, Maksim Eremeev, Jose Barrera, Abraham Bautista-Castillo, Eric Brown, David Ferrucci

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use

Yuhan Chen, Ang Lv, Ting-En Lin, Changyu Chen, Yuchuan Wu, Fei Huang, Yongbin Li, Rui Yan

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments ACL 2024 main

Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs

Fatemeh Shiri, Van Nguyen, Farhad Moghimifar, John Yoo, Gholamreza Haffari, Yuan-Fang Li

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation

Gauthier Guinet, Behrooz Omidvar-Tehrani, Anoop Deoras, Laurent Callot

专题命中 RAG评测 ：RAG(abstract)；分类 cs.IR、cs.CL

Comments Proceedings of the 41st International Conference on Machine Learning (ICML), 29 pages, 12 figures

Can large language models understand uncommon meanings of common words?

Jinyang Wu, Feihu Che, Xinxin Zheng, Shuai Zhang, Ruihan Jin, Shuai Nie, Pengpeng Shao, Jianhua Tao

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification

Haoqiang Kang, Juntong Ni, Huaxiu Yao

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss

Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Comments 11M tokens, fix qa3 min facts per task in Table 1

Experimenting with Large Language Models and vector embeddings in NASA SciX

Sergi Blanco-Cuaresma, Ioana Ciucă, Alberto Accomazzi, Michael J. Kurtz, Edwin A. Henneken, Kelly E. Lockhart, Felix Grezes, Thomas Allen, Golnaz Shapurian, Carolyn S. Grant, Donna M. Thompson, Timothy W. Hostetler, Matthew R. Templeton, Shinyi Chen, Jennifer Koch, Taylor Jacovich, Daniel Chivvis, Fernanda de Macedo Alves, Jean-Claude Paquin, Jennifer Bartlett, Mugdha Polimera, Stephanie Jarmak

专题命中 RAG评测 ：retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

Comments To appear in the proceedings of the 33th annual international Astronomical Data Analysis Software & Systems (ADASS XXXIII)

Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination

Haoqiang Kang, Xiao-Yang Liu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL、cs.AI

Chainpoll: A high efficacy method for LLM hallucination detection

Robert Friel, Atindriyo Sanyal

专题命中 RAG评测 ：retrieval augmented generation(abstract)；分类 cs.CL、cs.AI

A Study of Situational Reasoning for Traffic Understanding

Jiarui Zhang, Filip Ilievski, Kaixin Ma, Aravinda Kollaa, Jonathan Francis, Alessandro Oltramari

专题命中 RAG评测 ：dense retrieval(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 6 figures, 5 tables, camera ready version of SIGKDD 2023

When Is Complex Chunking Worth It? A Multi-Objective Evaluation of Chunking Methods at Scale

复杂分块何时值得？大规模分块方法的多目标评估

Laura Caspari, Kanishka Ghosh Dastidar, Michael Dinzinger, Jelena Mitrović, Michael Granitzer

专题命中 RAG评测 ：dense retrieval(abstract)；分类 cs.IR

AI总结 该研究针对长文档分块的多目标评估问题，在多语料库、多模型下对比八种分块策略，发现分块需兼顾检索性能与系统成本，应作为多目标设计决策。

TopoIntent: Compiling Security Intent into Executable, Compliance-Checked Network Topologies

TopoIntent：将安全意图编译为可执行、合规性检查的网络拓扑结构

Xiaokang Qu, Jianliang Ma, Zao Fan, Tianshu Chu, Tianlong Fan, Linyuan Lü

机构 * University of Science and Technology of China（中国科学技术大学） ； Topsec Technologies Group Inc.（天融信科技集团股份有限公司） 纠错

专题命中 RAG评测 ：vector search(abstract)；分类 cs.AI

AI总结 TopoIntent系统通过模式契约、向量搜索和分阶段融合将安全意图编译为合规可执行拓扑，经评估其修复后CIS满意度与ACL通过率均显著提升。

ContextSniper: AntTrail's Token-Efficient Code Memory for Repository-Level Program Repair

ContextSniper: AntTrail的令牌高效代码记忆用于仓库级程序修复

Chiwang Luk, Matin Mohammad Najafi, Zhifeng Jia, Wei Yang, Xiuchang Li, Jinwei Zhu, Yang Ren, Lei Chen, Gao Cong

机构 * Huawei（华为） ； HKUST(GZ)（香港科技大学（广州）） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 RAG评测 ：RAG(abstract)；分类 cs.AI

AI总结 提出ContextSniper，一种令牌高效的代码记忆层，通过精准证据选择、混合检索排序和意图感知上下文门控，在SWE-bench Lite上减少51.5%令牌使用和36.4%成本，修复率仅轻微下降。

Citation Grounding Measures the Oracle: Graph Coverage Determines Reported LLM Hallucination Rates in Law

引用溯源：通过法律引用图检测和减少LLM引用幻觉

Volodymyr Ovcharov

机构 * LEX AI LLC 纠错

专题命中 RAG评测 ：RAG(abstract_cn)；分类 cs.CL

AI总结 提出引用溯源（CG）指标，利用乌克兰法院判决的引用图（1.008亿判决，5.02亿边）检测LLM法律引用幻觉，并通过CG-DPO方法（基于真实判决构建偏好对）减少幻觉，在100个法律查询上CG为0.791-0.873，幻觉率13-21%。

Comments 21 pages, 4 figures, 5 tables. Substantially revised: title, framing and several v1 results changed. Adds a coverage sweep and a separability analysis; corrects the DPO configuration, the density-accuracy correlation and the qualitative examples. Code and data: https://huggingface.co/datasets/overthelex/citation-grounding-eval

DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction

DiagChain：用于评估大语言模型智能体基于证据的攻击链重构的诊断基准

Xuyang Liu, Yibin Han, Zhenwei Zhang, Kai Chang, Zhiwei Xu, Tian Qiu, Weixian Deng, Jiabao Gao, Xiaolin Peng, Hai Wan, Xibin Zhao

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.AI

AI总结 研究针对LLM智能体攻击链重构基准的不足，提出DiagChain基准与ECRAG方法，经6种LLM评估发现模型在证据整合与排序环节存在瓶颈，为网络安全智能体改进提供了诊断依据。

Zing: Social Mind for LLMs

Zing：大语言模型的社交智能

Zing Team, Ao Xiang, Bi Jingping, Chen Jiahui, Chen Lehan, Chen Yilin, Cheng Xueqi, Fan Yixing, Gan Kairong, Gao Haowen, Gao Jinhua, Gao Shuxuan, Gong Chang, Guo Jiafeng, Guo Ruijie, Han Zhouyu, He Guangfu, He Yichun, Jiang Shuo, Jing Shaoling, Jing Ya, Lei Chenhao, Lei Yan, Li Anqi, Li Chengao, Li Haoyu, Li Shitian, Liang Xinjian, Liu Zhaoge, Lyu Xingyu, Nie Zhuwei, Pang Liang, Quan Zeping, Shan Shiguang, Shen Huawei, Tang Xinran, Tian Feng, Wang Qian, Wang Ruiping, Wang Xiaohong, Xia Zaiyu, Xiao Yi, Xu Jiayuan, Xu Kehan, Xu Qianqian, Xu Tianyu, Xu Yongjun, Yang Haoming, Yang Jun, Yao Di, Yu Xiaoming, Zhang Futong, Zhang Jie, Zhang Shixuan, Zhang Yuxuan, Zhao Xinyu, Zhao Zhuoran, Zhong Yunfei, Zhu Shengyu

专题命中 RAG评测 ：RAG(abstract)；分类 cs.CL

AI总结 研究大语言模型社交智能，提出Zijing框架，含测量基准SoMBench、训练方法Zing及推理架构Actio，通过实验证明社交智能大语言模型在评估、内化和落地方面需协同发展。

Teachy Mini: Development and Preliminary Evaluation of a Knowledge-Based Generative Social Robot for Higher Education

Teachy Mini：用于高等教育的基于知识的生成式社交机器人的开发与初步评估

Stephan Vonschallen, Karim Kaufmann, Dominique Oberle, Friederike Eyssel, Theresa Schmiedel

机构 * Institute of Information Systems, Zurich University of Applied Sciences（苏黎世应用科学大学信息系统研究所） ； Center for Cognitive Interaction Technology, Bielefeld University（比勒费尔德大学认知交互技术中心） 纠错

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；分类 cs.AI

AI总结 研究针对生成式社交机器人在高等教育辅导中的风险，基于知识的设计要求开发Teachy Mini系统，经初步评估，该系统在负责任辅导行为及个性化等方面表现更佳，虽在部分方面与对照无显著差异，但显示出基于知识的设计对学习效果有积极作用。