语言大模型 / LLM - arXivDaily 专题

2502.15932 2025-02-25 cs.CL cs.CR 92%

CVE-LLM : Ontology-Assisted Automatic Vulnerability Evaluation Using Large Language Models

Rikhiya Ghosh, Hans-Martin von Stockhausen, Martin Schmitt, George Marica Vasile, Sanjeev Kumar Karn, Oladimeji Farri

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments arXiv admin note: substantial text overlap with arXiv:2407.14640

2502.02289 2025-02-05 cs.CL 92%

Evalita-LLM: Benchmarking Large Language Models on Italian

Bernardo Magnini, Roberto Zanoli, Michele Resta, Martin Cimmino, Paolo Albano, Marco Madeddu, Viviana Patti

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 42 pages, 1 figure, 32 tables

2411.00294 2024-11-05 cs.CL 92%

LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models

Kazi Ahmed Asif Fuad, Lizhong Chen

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 20 pages, 7 figures

2409.15310 2024-09-25 cs.LG cs.CV 92%

Visual Prompting in Multimodal Large Language Models: A Survey

Junda Wu, Zhehao Zhang, Yu Xia, Xintong Li, Zhaoyang Xia, Aaron Chang, Tong Yu, Sungchul Kim, Ryan A. Rossi, Ruiyi Zhang, Subrata Mitra, Dimitris N. Metaxas, Lina Yao, Jingbo Shang, Julian McAuley

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.LG

Comments 10 pages

2405.10825 2024-09-17 eess.SY cs.LG cs.SY 92%

Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities

Hao Zhou, Chengming Hu, Ye Yuan, Yufei Cui, Yili Jin, Can Chen, Haolun Wu, Dun Yuan, Li Jiang, Di Wu, Xue Liu, Charlie Zhang, Xianbin Wang, Jiangchuan Liu

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

2408.12787 2024-09-09 cs.CR cs.AI 92%

LLM-PBE: Assessing Data Privacy in Large Language Models

Qinbin Li, Junyuan Hong, Chulin Xie, Jeffrey Tan, Rachel Xin, Junyi Hou, Xavier Yin, Zhun Wang, Dan Hendrycks, Zhangyang Wang, Bo Li, Bingsheng He, Dawn Song

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2405.02363 2024-07-25 cs.CV cs.CL 92%

LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Model

Yulin Luo, Ruichuan An, Bocheng Zou, Yiming Tang, Jiaming Liu, Shanghang Zhang

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ECCV24 Camera Ready

2407.13244 2024-07-19 cs.CL cs.DB 92%

PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks

Alessandro Berti, Humam Kourani, Wil M. P. van der Aalst

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2407.08067 2024-07-12 cs.HC cs.AI 92%

On LLM Wizards: Identifying Large Language Models' Behaviors for Wizard of Oz Experiments

Jingchao Fang, Nikos Arechiga, Keiichi Namaoshi, Nayeli Bravo, Candice Hogan, David A. Shamma

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments To be published in ACM IVA 2024

Journal ref ACM International Conference on Intelligent Virtual Agents (IVA 2024), September 16-19, 2024, Glasgow, United Kingdom. ACM, New York, NY, USA

2407.00702 2024-07-02 cs.CL 92%

Scaling Technology Acceptance Analysis with Large Language Model (LLM) Annotation Systems

Pawel Robert Smolinski, Joseph Januszewicz, Jacek Winiarski

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments This is a preprint of a paper accepted for the 32nd International Conference on Information Systems Development (ISD 2024), Gdansk, Poland

2303.13809 2024-06-06 cs.CL 92%

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models

Qingyu Lu, Baopu Qiu, Liang Ding, Kanjian Zhang, Tom Kocmi, Dacheng Tao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL

Comments Findings of ACL 2024

2405.06524 2024-05-13 cs.CL 92%

Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts

Wenyu Huang, Guancheng Zhou, Mirella Lapata, Pavlos Vougiouklis, Sebastien Montella, Jeff Z. Pan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL

2405.04909 2024-05-09 cs.CV cs.AI 92%

Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models

Zhengxing Lan, Hongbo Li, Lingshan Liu, Bo Fan, Yisheng Lv, Yilong Ren, Zhiyong Cui

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2404.09329 2024-04-23 cs.CL 92%

Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments

Carlos Carrasco-Farre

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2403.16008 2024-03-26 cs.CL 92%

CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering

Hongbin Na

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted at COLING 2024

2307.07889 2024-02-07 cs.CL 92%

LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models

Adian Liusie, Potsawee Manakul, Mark J. F. Gales

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments To Appear at EACL 2024

2401.00284 2024-01-02 cs.CL 92%

Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Models

Maximilian Weber, Merle Reichardt

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL

2312.00909 2023-12-05 cs.IR cs.AI 92%

LLM-TAKE: Theme Aware Keyword Extraction Using Large Language Models

Reza Yousefi Maragheh, Chenhao Fang, Charan Chand Irugu, Parth Parikh, Jason Cho, Jianpeng Xu, Saranyan Sukumar, Malay Patel, Evren Korpeoglu, Sushant Kumar, Kannan Achan

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2310.19792 2023-10-31 cs.CL 92%

The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics

Christoph Leiter, Juri Opitz, Daniel Deutsch, Yang Gao, Rotem Dror, Steffen Eger

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(title,abstract)；分类 cs.CL

2310.15100 2023-10-24 cs.CL 92%

LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis

Shih-Chieh Dai, Aiping Xiong, Lun-Wei Ku

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP 2023 Findings

2404.01023 2026-07-01 cs.SE 91%

Large Language Model Evaluation Via Multi AI Agents: Preliminary results

通过多AI代理评估大型语言模型：初步结果

Zeeshan Rasheed, Muhammad Waseem, Kari Systä, Pekka Abrahamsson

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(summary_cn,abstract_cn)

AI总结本文提出多AI代理模型评估不同LLM性能，通过代码检索与验证，初步结果显示GPT-3.5 Turbo表现更优，未来将引入MBPP基准提升评估精度。

Comments 10 pages, 1 figure

Journal ref ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024

2404.01535 2026-05-11 cs.SE 91%

Assessing, Exploiting, and Mitigating Syntactic Robustness Failures in LLM-Based Code Generation

评估、利用和缓解基于LLM的代码生成中的语法鲁棒性故障

Laboni Sarker, Mara Downing, Achintya Desai, Tevfik Bultan

专题命中评测与基准：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；foundation model(journal_ref)

AI总结研究评估LLM在代码生成中语法鲁棒性，发现其在包含数学公式的提示下存在缺陷，提出预处理步骤提升鲁棒性，使鲁棒性从54.05%提升至74.42%。

Comments 12 pages, 12 figures. Attack strategies and more experimental evaluation is added. Result and big picture remains the same

Journal ref In Proceedings of the 2026 IEEE/ACM Third International Conference on AI Foundation Models and Software Engineering (FORGE'26), April 12-13, 2026, Rio de Janeiro, Brazil. ACM, New York, NY, USA, 12 pages

2502.09980 2026-02-17 cs.CV cs.RO 91%

V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models

V2V-LLM：基于多模态大语言模型的车与车协同自动驾驶

Hsu-kuang Chiu, Ryo Hachiuma, Chien-Yi Wang, Stephen F. Smith, Yu-Chiang Frank Wang, Min-Hung Chen

机构 * NVIDIA ； Carnegie Mellon University（卡内基梅隆大学）

专题命中评测与基准：LLM(title,abstract)；large language model(title,abstract)；language model(title,abstract)

AI总结本文提出基于多模态大语言模型的V2V-LLM，通过车与车协同感知提升自动驾驶安全性和性能。

Comments Accepted by ICRA 2026 (IEEE International Conference on Robotics and Automation). Project: https://eddyhkchiu.github.io/v2vllm.github.io/ Code: https://github.com/eddyhkchiu/V2V-LLM Dataset: https://huggingface.co/datasets/eddyhkchiu/V2V-GoT-QA

2511.02376 2025-12-23 cs.CL cs.AI cs.CR cs.LG 91%

AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models

AutoAdv：面向大语言模型多轮对抗性提示的自动化 jailbreaking

Aashray Reddy, Andrew Zagula, Nicholas Saban

机构 * University of California, Berkeley（加州大学伯克利分校）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(title)；分类 cs.CL、cs.AI、cs.LG

AI总结 AutoAdv通过多轮自适应机制实现高成功率的对抗性提示攻击，揭示当前安全机制在多轮对话中的脆弱性。

Comments Presented at NeurIPS 2025 Lock-LLM Workshop. Code is available at https://github.com/AAN-AutoAdv/AutoAdv

2608.12324 2026-08-14 cs.CY cs.AI cs.CL 新提交 91%

When AI Is Your Pastor: A Benchmark for Theological Triage and Pastoral Guidance in Large Language Models

当AI成为你的牧师：大型语言模型的神学分诊与牧灵指导基准测试

Alex Chao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(summary_cn,abstract_cn)；分类 cs.CL、cs.AI

AI总结本研究推出FMG-Bench基准测试，评估LLM在基督教神学分诊与牧灵指导场景的表现，发现结构化框架可提升模型表现与鲁棒性，且该基准仅为测量工具。

Comments Full paper. Code and dataset are available at https://github.com/FideAI/fmg-bench and https://huggingface.co/datasets/FideAI/fmg-bench

2602.00665 2026-08-04 cs.CL cs.AI 版本更新 91%

Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation

小语言模型能否处理上下文总结的多轮客户服务问答？一种合成数据驱动的比较评估

Lakshan Cooray, Deshan Sumanathilaka, Pattigadapa Venkatesh Raju

机构 * School of Computing, Informatics Institute of Technology（计算学院，信息学院技术研究所）； School of Mathematics and Computer Science, Swansea University（数学与计算机科学学院，萨默塞特大学）； R&D, Zame AI（Zame AI研发部）

专题命中评测与基准：language model(title,abstract)；small language model(title,abstract)；LLM(abstract,abstract_cn)；large language model(abstract)

AI总结本文通过合成数据评估小语言模型在多轮客户服务问答中的表现，发现部分模型接近大语言模型性能，但整体仍存在对话连续性和上下文对齐的挑战。

Comments Published at Frontiers in Artificial Intelligence, Natural Language Processing Section

Journal ref Frontiers in Artificial Intelligence, 9:1804284, 2026

2605.19723 2026-07-09 cs.CL cs.AI 版本更新 91%

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

大型语言模型中的数学推理：基准测试、架构、评估与开放挑战

Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima

机构 * organization= School of Electrical Engineering ； Computer Science, National University of Science ； organization= School of Computing, Data ； Mathematical Sciences, Western Sydney University, Indonesia ； organization= Department of Communication, Quality Management ； Information Systems, Mid Sweden University, Östersund Campus, Sweden

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,abstract_cn)；pretraining(abstract)

AI总结本文综述了大型语言模型在数学推理方面的最新进展，通过分析数据集、架构、训练策略和评估协议，探讨了数学推理的基准测试、架构设计、评估方法以及未来的研究挑战。

2607.01829 2026-07-03 cs.AI cs.CL 新提交 91%

Pre-Flight: A Benchmark for Evaluating Large Language Models on Aviation Operational Knowledge

Pre-Flight: 评估大型语言模型航空运营知识的基准

Alex Brooker, Tim Hughes

机构 * Airside Labs, London, United Kingdom（Airside Labs，伦敦，英国）； Mahino Research, Christchurch, New Zealand（Mahino Research，基督城，新西兰）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(summary_cn,abstract_cn)；分类 cs.CL、cs.AI

AI总结提出Pre-Flight基准，包含300道多选题，评估LLM在航空运营知识上的表现，发现最强模型准确率82.7%，低于专家水平的95%，表明领域特定评估的必要性。

Comments 9 pages, 1 figure, 2 tables. Benchmark available in inspect_evals (UKGovernmentBEIS/inspect_evals)

2606.31718 2026-07-01 cs.CL cs.AI 新提交 91%

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

跨语言关系抽取与大语言模型：罗马尼亚语的零样本、少样本和微调评估

Dragos-Mitrut Vasile, Elena-Simona Apostol, Stefan-Adrian Toma, Adrian Paschke, Ciprian-Octavian Truica

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,abstract_cn)；prompting(abstract)

AI总结通过自动翻译数据集结合大语言模型推理，研究罗马尼亚语的跨语言关系抽取，评估Gemma 4 31B在零样本、少样本和QLoRA微调下的性能，并与四种编码器基线对比。

2606.21820 2026-06-23 cs.SI cs.AI cs.CL 新提交 91%

Generating Public Health Responses using Survey-Augmented Large Language Models

使用调查增强的大语言模型生成公共卫生响应

Leonardo Marciaga, Thuyen Pham, Julia Rezvani, Alina Hyk, Chunyang Liao, Konstantinos Mitsopoulos, Raffaele Vardavas

机构 * Hawk.illinoistech.edu（伊利诺伊理工学院）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,abstract_cn)；prompting(abstract)

AI总结研究利用大语言模型生成合成调查数据，以模拟真实人群在流行病中的健康决策行为，发现合成数据能复现人口统计和信念分布，但难以捕捉因素间的协变关系，不能替代真实调查。

Comments 24 pages, 6 figures

1. 评测与基准 31794 篇

CVE-LLM : Ontology-Assisted Automatic Vulnerability Evaluation Using Large Language Models

Evalita-LLM: Benchmarking Large Language Models on Italian

LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models

Visual Prompting in Multimodal Large Language Models: A Survey

Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities

LLM-PBE: Assessing Data Privacy in Large Language Models

LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Model

PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks

On LLM Wizards: Identifying Large Language Models' Behaviors for Wizard of Oz Experiments

Scaling Technology Acceptance Analysis with Large Language Model (LLM) Annotation Systems

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models

Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts

Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models

Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments

CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering

LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models

Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Models

LLM-TAKE: Theme Aware Keyword Extraction Using Large Language Models

The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics

LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis

Large Language Model Evaluation Via Multi AI Agents: Preliminary results

Assessing, Exploiting, and Mitigating Syntactic Robustness Failures in LLM-Based Code Generation

V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models

AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models

When AI Is Your Pastor: A Benchmark for Theological Triage and Pastoral Guidance in Large Language Models

Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

Pre-Flight: A Benchmark for Evaluating Large Language Models on Aviation Operational Knowledge

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

Generating Public Health Responses using Survey-Augmented Large Language Models