语言大模型 / LLM

Large Language Models for Data Annotation and Synthesis: A Survey

Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, Huan Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2024 Main

An Extensive Evaluation of Factual Consistency in Large Language Models for Data-to-Text Generation

Joy Mahapatra, Utpal Garain

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 15 pages

Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT's Effectiveness with Different Settings and Inputs

Mike Thelwall

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Large language models surpass human experts in predicting neuroscience results

Xiaoliang Luo, Akilles Rechardt, Guangzhi Sun, Kevin K. Nejad, Felipe Yáñez, Bati Yilmaz, Kangjoo Lee, Alexandra O. Cohen, Valentina Borghesani, Anton Pashkov, Daniele Marinazzo, Jonathan Nicholas, Alessandro Salatiello, Ilia Sucholutsky, Pasquale Minervini, Sepehr Razavi, Roberta Rocca, Elkhan Yusifov, Tereza Okalova, Nianlong Gu, Martin Ferianc, Mikail Khona, Kaustubh R. Patil, Pui-Shee Lee, Rui Mata, Nicholas E. Myers, Jennifer K Bizley, Sebastian Musslick, Isil Poyraz Bilgin, Guiomar Niso, Justin M. Ales, Michael Gaebler, N Apurva Ratan Murty, Leyla Loued-Khenissi, Anna Behler, Chloe M. Hall, Jessica Dafflon, Sherry Dongqi Bao, Bradley C. Love

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments The latest version of this paper has been published at Nature Human Behaviour, please see https://www.nature.com/articles/s41562-024-02046-9

LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models

Haitao Li, You Chen, Qingyao Ai, Yueyue Wu, Ruizhe Zhang, Yiqun Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments NeurIPs 2024

AutoLLM-CARD: Towards a Description and Landscape of Large Language Models

Shengwei Tian, Lifeng Han, Goran Nenadic

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ongoing work, technical report

SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model

Christopher Nguyen, William Nguyen, Atsushi Suzuki, Daisuke Oku, Hong An Phan, Sang Dinh, Zooey Nguyen, Anh Ha, Shruti Raghavan, Huy Vo, Thang Nguyen, Lan Nguyen, Yoshikuni Hirayama

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments On-going work

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, Ting Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by ACM Transactions on Information Systems (TOIS)

Evaluating Gender Bias in Large Language Models

Michael Döll, Markus Döhring, Andreas Müller

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments 13 pages, 12 figures, 1 table

Beyond Static Tools: Evaluating Large Language Models for Cryptographic Misuse Detection

Zohaib Masood, Miguel Vargas Martin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models

Linyi Li, Shijie Geng, Zhenwen Li, Yibo He, Hao Yu, Ziyue Hua, Guanghan Ning, Siwei Wang, Tao Xie, Hongxia Yang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments 31 pages. Appear at NeurIPS 2024 Datasets and Benchmarks track. Project website: https://infi-coder.github.io/infibench

Responsible AI in Construction Safety: Systematic Evaluation of Large Language Models and Prompt Engineering

Farouq Sammour, Jia Xu, Xi Wang, Mo Hu, Zhenyu Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 29 pages, 5 figures

A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look

Shivani Upadhyay, Ronak Pradeep, Nandan Thakur, Daniel Campos, Nick Craswell, Ian Soboroff, Hoa Trang Dang, Jimmy Lin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Entity Extraction from High-Level Corruption Schemes via Large Language Models

Panagiotis Koletsis, Panagiotis-Konstantinos Gemos, Christos Chronis, Iraklis Varlamis, Vasilis Efthymiou, Georgios Th. Papadopoulos

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models

Chani Jung, Dongkwan Kim, Jiho Jin, Jiseon Kim, Yeon Seonwoo, Yejin Choi, Alice Oh, Hyunwoo Kim

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Towards a Unified View of Preference Learning for Large Language Models: A Survey

Bofei Gao, Feifan Song, Yibo Miao, Zefan Cai, Zhe Yang, Liang Chen, Helan Hu, Runxin Xu, Qingxiu Dong, Ce Zheng, Shanghaoran Quan, Wen Xiao, Ge Zhang, Daoguang Zan, Keming Lu, Bowen Yu, Dayiheng Liu, Zeyu Cui, Jian Yang, Lei Sha, Houfeng Wang, Zhifang Sui, Peiyi Wang, Tianyu Liu, Baobao Chang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 23 pages, 6 figures

Large Language Models for Generative Information Extraction: A Survey

Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, Yang Wang, Enhong Chen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments The article has been accepted by Frontiers of Computer Science (FCS), with the DOI: {10.1007/s11704-024-40555-y}. You can cite the FCS version

ESC-Eval: Evaluating Emotion Support Conversations in Large Language Models

Haiquan Zhao, Lingyu Li, Shisong Chen, Shuqi Kong, Jiaan Wang, Kexin Huang, Tianle Gu, Yixu Wang, Wang Jian, Dandan Liang, Zhixu Li, Yan Teng, Yanghua Xiao, Yingchun Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments EMNLP 2024

Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithms

He Yu, Jing Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Benchmarking Large Language Models for Image Classification of Marine Mammals

Yijiashun Qi, Shuzhang Cai, Zunduo Zhao, Jiaming Li, Yanbin Lin, Zhiqiang Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ICKG 2024

A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning

Shengjie Sun, Runze Liu, Jiafei Lyu, Jing-Wen Yang, Liangpeng Zhang, Xiu Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models

Virginia K. Felkner, Ho-Chun Herbert Chang, Eugene Jang, Jonathan May

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to ACL 2023 (main conference). This version corrects a bug found in evaluation code after publication. General findings have not changed, but tables 5 and 6 and figure 1 have been corrected

Better to Ask in English: Evaluation of Large Language Models on English, Low-resource and Cross-Lingual Settings

Krishno Dey, Prerona Tarannum, Md. Arid Hasan, Imran Razzak, Usman Naseem

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Pragmatic Competence Evaluation of Large Language Models for the Korean Language

Dojun Park, Jiwoo Lee, Hyeyun Jeong, Seohyun Park, Sungeun Lee

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments 38th Pacific Asia Conference on Language, Information and Computation

The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers

Hussein Mozannar, Valerie Chen, Mohammed Alsobay, Subhro Das, Sebastian Zhao, Dennis Wei, Manish Nagireddy, Prasanna Sattigeri, Ameet Talwalkar, David Sontag

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Detecting Bias and Enhancing Diagnostic Accuracy in Large Language Models for Healthcare

Pardis Sadat Zahraei, Zahra Shakeri

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

Jiahui Gao, Renjie Pi, Tianyang Han, Han Wu, Lanqing Hong, Lingpeng Kong, Xin Jiang, Zhenguo Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 10 pages, COLM-2024

Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems

Ning Lu, Qian Xie, Hao Zhang, Wenyi Fang, Yang Zheng, Zheng Hu, Jiantao Ma

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments To be published in: IEEE International Symposium on Software Reliability Engineering (ISSRE2024) workshop

Can AI Relate: Testing Large Language Model Response for Mental Health Support

Saadia Gabriel, Isha Puri, Xuhai Xu, Matteo Malgaroli, Marzyeh Ghassemi

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments EMNLP 2024 Findings

UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction

Yansong Ning, Hao Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments NeurIPS 2024