语言大模型 / LLM - arXivDaily 专题

2402.00262 2024-02-02 cs.AI 89%

Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspective

Qun Ma, Xiao Xue, Deyu Zhou, Xiangning Yu, Donghua Liu, Xuwen Zhang, Zihan Zhao, Yifan Shen, Peilin Ji, Juanjuan Li, Gang Wang, Wanpeng Ma

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2401.16745 2024-01-31 cs.CL 89%

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models

Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Code and data are available at https://github.com/KwanWaiChung/MT-Eval

2401.15927 2024-01-30 cs.CL 89%

E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models

Jinchang Hou, Chang Ao, Haihong Wu, Xiangtao Kong, Zhigang Zheng, Daijia Tang, Chengming Li, Xiping Hu, Ruifeng Xu, Shiwen Ni, Min Yang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

2310.14703 2024-01-30 cs.CL 89%

Establishing Vocabulary Tests as a Benchmark for Evaluating Large Language Models

Gonzalo Martínez, Javier Conde, Elena Merino-Gómez, Beatriz Bermúdez-Margaretto, José Alberto Hernández, Pedro Reviriego, Marc Brysbaert

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2310.03266 2024-01-18 cs.LG 89%

UniPredict: Large Language Models are Universal Tabular Classifiers

Ruiyu Wang, Zifeng Wang, Jimeng Sun

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

2401.08206 2024-01-17 cs.IR cs.CL 89%

Generative Multi-Modal Knowledge Retrieval with Large Language Models

Xinwei Long, Jiali Zeng, Fandong Meng, Zhiyuan Ma, Kaiyan Zhang, Bowen Zhou, Jie Zhou

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to AAAI 2024

2312.14033 2024-01-17 cs.CL 89%

T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step

Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, Feng Zhao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Project: https://open-compass.github.io/T-Eval

2401.02208 2024-01-05 cs.CL 89%

DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models

Songbo Hu, Xiaobin Wang, Zhangdie Yuan, Anna Korhonen, Ivan Vulić

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 17 pages, 7 tables, 9 figures

2401.01711 2024-01-04 cs.CL cs.IR 89%

Evaluating Large Language Models in Semantic Parsing for Conversational Question Answering over Knowledge Graphs

Phillip Schneider, Manuel Klettner, Kristiina Jokinen, Elena Simperl, Florian Matthes

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Accepted to ICAART 2024

2401.00690 2024-01-02 cs.CL 89%

Benchmarking Large Language Models on Controllable Generation under Diversified Instructions

Yihan Chen, Benfeng Xu, Quan Wang, Yi Liu, Zhendong Mao

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to AAAI 2024

2308.15645 2023-12-29 cs.PL cs.AI cs.SE 89%

AskIt: Unified Programming Interface for Programming with Large Language Models

Katsumi Okuda, Saman Amarasinghe

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments To be published in 2024 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

2312.15746 2023-12-27 cs.IR cs.AI 89%

Large Language Models are Not Stable Recommender Systems

Tianhui Ma, Yuan Cheng, Hengshu Zhu, Hui Xiong

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2312.07488 2023-12-22 cs.CV cs.AI cs.RO 89%

LMDrive: Closed-Loop End-to-End Driving with Large Language Models

Hao Shao, Yuxuan Hu, Letian Wang, Steven L. Waslander, Yu Liu, Hongsheng Li

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments project page: https://hao-shao.com/projects/lmdrive.html

2310.07521 2023-12-19 cs.CL 89%

Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, Yue Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 62 pages; 300+ references

2311.03731 2023-12-15 cs.CL 89%

A Survey of Large Language Models Attribution

Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, Min Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2312.06002 2023-12-12 cs.CL 89%

Large Language Models on Lexical Semantic Change Detection: An Evaluation

Ruiyu Wang, Matthew Choi

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

2312.04019 2023-12-08 q-bio.BM cs.AI 89%

Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models

Yijie Zhang, Zhangyang Gao, Cheng Tan, Stan Z. Li

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2211.08412 2023-12-05 cs.CL 89%

Evaluating the Factual Consistency of Large Language Models Through News Summarization

Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, Colin Raffel

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2312.00164 2023-12-04 cs.CY cs.AI 89%

Towards Accurate Differential Diagnosis with Large Language Models

Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkarni, Le Hou, Yong Cheng, Yun Liu, S Sara Mahdavi, Sushant Prakash, Anupam Pathak, Christopher Semturs, Shwetak Patel, Dale R Webster, Ewa Dominowska, Juraj Gottweis, Joelle Barral, Katherine Chou, Greg S Corrado, Yossi Matias, Jake Sunshine, Alan Karthikesalingam, Vivek Natarajan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2311.18658 2023-12-01 cs.CL 89%

ArcMMLU: A Library and Information Science Benchmark for Large Language Models

Shitou Zhang, Zuchao Li, Xingshen Liu, Liming Yang, Ping Wang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2311.16103 2023-11-29 cs.CV cs.AI 89%

Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models

Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, Li Yuan

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments Benchmark is available at https://github.com/PKU-YuanGroup/Video-Bench

2310.00741 2023-11-29 cs.CL 89%

FELM: Benchmarking Factuality Evaluation of Large Language Models

Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, Junxian He

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by NeurIPS 2023 Track on Datasets and Benchmarks

2311.13053 2023-11-23 cs.CL 89%

Beyond Text: Unveiling Multimodal Proficiency of Large Language Models with MultiAPI Benchmark

Xiao Liu, Jianfeng Lin, Jiawei Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Work in Progress

2311.12338 2023-11-22 cs.IR cs.AI 89%

A Survey on Large Language Models for Personalized and Explainable Recommendations

Junyi Chen

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2311.09517 2023-11-17 cs.CL 89%

GEE! Grammar Error Explanation with Large Language Models

Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Kevin Gimpel, Mohit Iyyer

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Preprint, 24 pages, code and data available in https://github.com/Yixiao-Song/GEE-with-LLMs

2311.06838 2023-11-14 cs.CL 89%

GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect

Chengguang Gan, Qinghao Zhang, Tatsunori Mori

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 10 pages, 6 figures

2311.06697 2023-11-14 cs.CL 89%

Trusted Source Alignment in Large Language Models

Vasilisa Bashlovkina, Zhaobin Kuang, Riley Matthews, Edward Clifford, Yennie Jun, William W. Cohen, Simon Baumgartner

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2308.05345 2023-11-14 cs.LG cs.AR 89%

RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model

Yao Lu, Shang Liu, Qijun Zhang, Zhiyao Xie

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Journal ref Asia and South Pacific Design Automation Conference (ASP-DAC) 2024

2307.03025 2023-11-14 cs.CL 89%

Style Over Substance: Evaluation Biases for Large Language Models

Minghao Wu, Alham Fikri Aji

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Work in progress, 15 pages, 5 tables, 12 figures

2309.00608 2023-11-10 cs.SE cs.LG cs.PL 89%

Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair

Yuxiang Wei, Chunqiu Steven Xia, Lingming Zhang

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments Paper accepted at ESEC/FSE 2023

1. 评测与基准 32034 篇

Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspective

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models

E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models

Establishing Vocabulary Tests as a Benchmark for Evaluating Large Language Models

UniPredict: Large Language Models are Universal Tabular Classifiers

Generative Multi-Modal Knowledge Retrieval with Large Language Models

T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step

DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models

Evaluating Large Language Models in Semantic Parsing for Conversational Question Answering over Knowledge Graphs

Benchmarking Large Language Models on Controllable Generation under Diversified Instructions

AskIt: Unified Programming Interface for Programming with Large Language Models

Large Language Models are Not Stable Recommender Systems

LMDrive: Closed-Loop End-to-End Driving with Large Language Models

Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity

A Survey of Large Language Models Attribution

Large Language Models on Lexical Semantic Change Detection: An Evaluation

Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models

Evaluating the Factual Consistency of Large Language Models Through News Summarization

Towards Accurate Differential Diagnosis with Large Language Models

ArcMMLU: A Library and Information Science Benchmark for Large Language Models

Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models

FELM: Benchmarking Factuality Evaluation of Large Language Models

Beyond Text: Unveiling Multimodal Proficiency of Large Language Models with MultiAPI Benchmark

A Survey on Large Language Models for Personalized and Explainable Recommendations

GEE! Grammar Error Explanation with Large Language Models

GIELLM: Japanese General Information Extraction Large Language Model Utilizing Mutual Reinforcement Effect

Trusted Source Alignment in Large Language Models

RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model

Style Over Substance: Evaluation Biases for Large Language Models

Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair