语言大模型 / LLM

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models

Yu Yuan, Lili Zhao, Kai Zhang, Guangting Zheng, Qi Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Roadmap towards Superhuman Speech Understanding using Large Language Models

Fan Bu, Yuhao Zhang, Xidong Wang, Benyou Wang, Qun Liu, Haizhou Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.AI

LFOSum: Summarizing Long-form Opinions with Large Language Models

Mir Tafseer Nayeem, Davood Rafiei

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs

Divyanshu Kumar, Umang Jain, Sahil Agarwal, Prashanth Harshangi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Enterprise Benchmarks for Large Language Model Evaluation

Bing Zhang, Mikio Takeuchi, Ryo Kawahara, Shubhi Asthana, Md. Maruf Hossain, Guang-Jie Ren, Kate Soule, Yada Zhu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

On the Capacity of Citation Generation by Large Language Models

Haosheng Qian, Yixing Fan, Ruqing Zhang, Jiafeng Guo

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted by CCIR 2024

AuditWen:An Open-Source Large Language Model for Audit

Jiajia Huang, Haoran Zhu, Chao Xu, Tianming Zhan, Qianqian Xie, Jimin Huang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 18 pages,1 figures

The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models

Siyang Liu, Trish Maturi, Bowen Yi, Siqi Shen, Rada Mihalcea

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 5 pages

Journal ref The 2024 Conference on Empirical Methods in Natural Language Processing

The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models

Subhankar Maity, Aniket Deroy

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Book Chapter (Under Review)

Towards a Benchmark for Large Language Models for Business Process Management Tasks

Kiran Busch, Henrik Leopold

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Submitted to HICSS (June 15, 2024)

A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models

Cong Guo, Feng Cheng, Zhixu Du, James Kiessling, Jonathan Ku, Shiyu Li, Ziru Li, Mingyuan Ma, Tergel Molom-Ochir, Benjamin Morris, Haoxuan Shan, Jingwei Sun, Yitu Wang, Chiyue Wei, Xueying Wu, Yuhao Wu, Hao Frank Yang, Jingyang Zhang, Junyao Zhang, Qilin Zheng, Guanglei Zhou, Hai, Li, Yiran Chen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments Accepted by IEEE Circuits and Systems Magazine

Representing the Under-Represented: Cultural and Core Capability Benchmarks for Developing Thai Large Language Models

Dahyun Kim, Sukyung Lee, Yungi Kim, Attapol Rutherford, Chanjun Park

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models

Jiasheng Zheng, Boxi Cao, Zhengzhao Ma, Ruotong Pan, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments We release benchmark at https://github.com/jszheng21/RACE and leaderboard at https://huggingface.co/spaces/jszheng/RACE_leaderboard

Evalverse: Unified and Accessible Library for Large Language Model Evaluation

Jihoo Kim, Wonho Song, Dahyun Kim, Yunsu Kim, Yungi Kim, Chanjun Park

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Demo Track

A Multi-Expert Large Language Model Architecture for Verilog Code Generation

Bardia Nadimi, Hao Zheng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models

Stephen R. Pfohl, Heather Cole-Lewis, Rory Sayres, Darlene Neal, Mercy Asiedu, Awa Dieng, Nenad Tomasev, Qazi Mamunur Rashid, Shekoofeh Azizi, Negar Rostamzadeh, Liam G. McCoy, Leo Anthony Celi, Yun Liu, Mike Schaekermann, Alanna Walton, Alicia Parrish, Chirag Nagpal, Preeti Singh, Akeiylah Dewitt, Philip Mansfield, Sushant Prakash, Katherine Heller, Alan Karthikesalingam, Christopher Semturs, Joelle Barral, Greg Corrado, Yossi Matias, Jamila Smith-Loud, Ivor Horn, Karan Singhal

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Journal ref Nature Medicine (2024)

HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language Models

Huy Nghiem, Hal Daumé

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 Findings

Evaluating Large Language Models in Analysing Classroom Dialogue

Yun Long, Haifeng Luo, Yu Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models

Yuxiang Zhang, Jing Chen, Junjie Wang, Yaxin Liu, Cheng Yang, Chufan Shi, Xinyu Zhu, Zihao Lin, Hanwen Wan, Yujiu Yang, Tetsuya Sakai, Tian Feng, Hayato Yamana

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Enhancing Tool Retrieval with Iterative Feedback from Large Language Models

Qiancheng Xu, Yongqi Li, Heming Xia, Wenjie Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Cultural Bias and Cultural Alignment of Large Language Models

Yan Tao, Olga Viberg, Ryan S. Baker, Rene F. Kizilcec

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Journal ref PNAS Nexus, Volume 3, Issue 9, September 2024, pgae346

Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey

Haochen Li, Jonathan Leung, Zhiqi Shen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments An up-to-date resource including papers and tasks is maintained at https://github.com/Alex-HaochenLi/Goal-oriented-Prompt-Engineering

Exploring Graph Structure Comprehension Ability of Multimodal Large Language Models: Case Studies

Zhiqiang Zhong, Davide Mottin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Assessing Adversarial Robustness of Large Language Models: An Empirical Study

Zeyu Yang, Zhao Meng, Xiaochen Zheng, Roger Wattenhofer

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Oral presentation at KDD 2024 GenAI Evaluation workshop

Ontology-Free General-Domain Knowledge Graph-to-Text Generation Dataset Synthesis using Large Language Model

Daehee Kim, Deokhyung Kang, Sangwon Ryu, Gary Geunbae Lee

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 9 figures

Large Language Models Synergize with Automated Machine Learning

Jinglue Xu, Jialong Li, Zhen Liu, Nagar Anthel Venkatesh Suryanarayanan, Guoyuan Zhou, Jia Guo, Hitoshi Iba, Kenji Tei

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments published at TMLR

PointLLM: Empowering Large Language Models to Understand Point Clouds

Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, Dahua Lin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments ECCV 2024 Oral Camera Ready. This version includes clearer writing and additional experimental results compared to previous versions. Project page: https://runsenxu.com/projects/PointLLM

Multi-Programming Language Ensemble for Code Generation in Large Language Model

Tengfei Xue, Xuefeng Li, Tahir Azim, Roman Smirnov, Jianhui Yu, Arash Sadrieh, Babak Pahlavan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Code available at https://github.com/NinjaTech-AI/MPLE

Eliciting Informative Text Evaluations with Large Language Models

Yuxuan Lu, Shengwei Xu, Yichi Zhang, Yuqing Kong, Grant Schoenebeck

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted by the Twenty-Fifth ACM Conference on Economics and Computation (EC'24)

A Survey for Large Language Models in Biomedicine

Chong Wang, Mengyao Li, Junjun He, Zhongruo Wang, Erfan Darzi, Zan Chen, Jin Ye, Tianbin Li, Yanzhou Su, Jing Ke, Kaili Qu, Shuxin Li, Yi Yu, Pietro Liò, Tianyun Wang, Yu Guang Wang, Yiqing Shen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI