RAG / 检索增强生成

MultiTEND: A Multilingual Benchmark for Natural Language to NoSQL Query Translation

Zhiqian Qin, Yuanfeng Song, Jinwei Lu, Yuanwei Song, Shuaimin Li, Chen Jason Zhang

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Panza: Design and Analysis of a Fully-Local Personalized Text Writing Assistant

Armand Nicolicioiu, Eugenia Iofinova, Andrej Jovanovic, Eldar Kurtic, Mahdi Nikdan, Andrei Panferov, Ilia Markov, Nir Shavit, Dan Alistarh

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Panza is available at https://github.com/IST-DASLab/PanzaMail

Efficient Knowledge Feeding to Language Models: A Novel Integrated Encoder-Decoder Architecture

S Santosh Kumar, Rishi Gottimukkala, Supriya Devidutta, Karthikeyan S

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.CL

Comments Submitted to ACM TIST journal: under revision stage, 8 pages, 2 figures

Reading with Intent -- Neutralizing Intent

Benjamin Reichman, Adar Avsian, Larry Heck

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Tanuja Ganu, Sunayana Sitaram, Kalika Bali, Akshay Nambi

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:2305.17740

Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs

Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Sunayana Sitaram, Kalika Bali, Tanuja Ganu, Akshay Nambi

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Journal ref COLING 2025

Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization

Yanhu Wang, Muhammad Muzammil Afzal, Zhengyang Li, Jie Zhou, Chenyuan Feng, Shuaishuai Guo, Tony Q. S. Quek

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders

Xiaofeng Zhu, Jaya Krishna Mandivarapu

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Journal ref EMNLP CustomNLP4U 2024

Revolutionizing Database Q&A with Large Language Models: Comprehensive Benchmark and Evaluation

Yihang Zheng, Bo Li, Zhenghao Lin, Yi Luo, Xuanhe Zhou, Chen Lin, Jinsong Su, Guoliang Li, Shifu Li

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI、cs.DB

Comments 12 pages

A Novel Approach to Eliminating Hallucinations in Large Language Model-Assisted Causal Discovery

Grace Sng, Yanming Zhang, Klaus Mueller

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

LEAF: Learning and Evaluation Augmented by Fact-Checking to Improve Factualness in Large Language Models

Hieu Tran, Junda Wang, Yujan Ting, Weijing Huang, Terrence Chen

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 9 figures

Assessing the Performance of Human-Capable LLMs -- Are LLMs Coming for Your Job?

John Mavi, Nathan Summers, Sergio Coronado

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs

Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, Mike Qi, Ruixuan Tu, Chenyu Xu, Matthew Gonzales, Ofer Mendelevitch, Amin Ahmad

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models

Yeeun Kim, Young Rok Choi, Eunkyung Choi, Jinhwan Choi, Hai Jin Park, Wonseok Hwang

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 Findings

Increasing the Difficulty of Automatically Generated Questions via Reinforcement Learning with Synthetic Preference

William Thorne, Ambrose Robinson, Bohua Peng, Chenghua Lin, Diana Maynard

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments is to be published in NLP4DH 2024

Benchmarking Cognitive Domains for LLMs: Insights from Taiwanese Hakka Culture

Chen-Chi Chang, Ching-Yuan Chen, Hung-Shin Lee, Chih-Cheng Lee

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted to O-COCOSDA 2024

Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis

Matteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

VERA: Validation and Evaluation of Retrieval-Augmented Systems

Tianyu Ding, Adi Banerjee, Laurent Mombaerts, Yunhong Li, Tarik Borogovac, Juan Pablo De la Cruz Weinstein

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

Comments Accepted in Workshop on Evaluation and Trustworthiness of Generative AI Models, KDD 2024

The Role of Large Language Models in Musicology: Are We Ready to Trust the Machines?

Pedro Ramoneda, Emilia Parada-Cabaleiro, Benno Weck, Xavier Serra

专题命中 RAG评测 ：retrieval augmented generation(abstract)；retrieval-augmented generation(abstract)；分类 cs.CL、cs.AI

Evaluating ChatGPT on Nuclear Domain-Specific Data

Muhammad Anwar, Mischa de Costa, Issam Hammad, Daniel Lau

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Journal ref 43rd Annual CNS Conference and the 48th Annual CNS/CNA Student Conference Sheraton Cavalier Saskatoon Hotel, Saskatoon, SK, Canada, June 16-19, 2024

Plan with Code: Comparing approaches for robust NL to DSL generation

Nastaran Bassamzadeh, Chhaya Methani

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 1 figure, 5 tables. arXiv admin note: substantial text overlap with arXiv:2407.02742

Lynx: An Open Source Hallucination Evaluation Model

Selvan Sunitha Ravi, Bartosz Mielczarek, Anand Kannappan, Douwe Kiela, Rebecca Qian

专题命中 RAG评测 ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networks

Pranshav Gajjar, Vijay K. Shah

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

How do you know that? Teaching Generative Language Models to Reference Answers to Biomedical Questions

Bojana Bašaragin, Adela Ljajić, Darija Medvecki, Lorenzo Cassano, Miloš Košprdić, Nikola Milošević

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments Accepted at BioNLP Workshop 2024, colocated with ACL 2024

BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science

Xinna Lin, Siqi Ma, Junjie Shan, Xiaojing Zhang, Shell Xu Hu, Tiannan Guo, Stan Z. Li, Kaicheng Yu

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Generation of Asset Administration Shell with Large Language Model Agents: Toward Semantic Interoperability in Digital Twins in the Context of Industry 4.0

Yuchen Xia, Zhewen Xiao, Nasser Jazdi, Michael Weyrich

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

Comments Published in IEEE Access

WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia

Yufang Hou, Alessandra Pascale, Javier Carnerero-Cano, Tigran Tchrakian, Radu Marinescu, Elizabeth Daly, Inkit Padhi, Prasanna Sattigeri

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

FinBen: A Holistic Financial Benchmark for Large Language Models

Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Kuang, Chenhan Yuan, Kailai Yang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, Guojun Xiong, Zhiyang Deng, Yuechen Jiang, Zhiyuan Yao, Haohang Li, Yangyang Yu, Gang Hu, Jiajia Huang, Xiao-Yang Liu, Alejandro Lopez-Lira, Benyou Wang, Yanzhao Lai, Hao Wang, Min Peng, Sophia Ananiadou, Jimin Huang

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 11 figures

Designing an Evaluation Framework for Large Language Models in Astronomy Research

John F. Wu, Alina Hyk, Kiera McCormick, Christine Ye, Simone Astarita, Elina Baral, Jo Ciuca, Jesse Cranney, Anjalie Field, Kartheik Iyer, Philipp Koehn, Jenn Kotler, Sandor Kruk, Michelle Ntampaka, Charles O'Neill, Joshua E. G. Peek, Sanjib Sharma, Mikaeel Yunus

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR、cs.AI

Comments 7 pages, 3 figures. Code available at https://github.com/jsalt2024-evaluating-llms-for-astronomy/astro-arxiv-bot

Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variability

Jinwei Lu, Yuanfeng Song, Haodi Zhang, Chen Zhang, Raymond Chi-Wing Wong

专题命中 RAG评测 ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL、cs.AI