语言大模型 / LLM

Evaluating Large Language Models: A Comprehensive Survey

Zishan Guo, Renren Jin, Chuang Liu, Yufei Huang, Dan Shi, Supryadi, Linhao Yu, Yan Liu, Jiaxuan Li, Bojian Xiong, Deyi Xiong

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 111 pages

Generating Valid and Natural Adversarial Examples with Large Language Models

Zimu Wang, Wei Wang, Qi Chen, Qiufeng Wang, Anh Nguyen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Submitted to the IEEE for possible publication

Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation

Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, Jingren Zhou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments We have released code on https://github.com/BeachWang/DAIL-SQL

Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values

Jing Yao, Xiaoyuan Yi, Xiting Wang, Yifan Gong, Xing Xie

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

InCA: Rethinking In-Car Conversational System Assessment Leveraging Large Language Models

Ken E. Friedl, Abbas Goher Khan, Soumya Ranjan Sahoo, Md Rashad Al Hasan Rony, Jana Germies, Christian Süß

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions

Noam Rotstein, David Bensaid, Shaked Brody, Roy Ganz, Ron Kimmel

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases

Juan Sequeda, Dean Allemang, Bryon Jacob

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 34 pages

Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks

Pouya Pezeshkpour, Hayate Iso, Thom Lake, Nikita Bhutani, Estevam Hruschka

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Context Unlocks Emotions: Text-based Emotion Classification Dataset Auditing with Large Language Models

Daniel Yang, Aditya Kommineni, Mohammad Alshehri, Nilamadhab Mohanty, Vedant Modi, Jonathan Gratch, Shrikanth Narayanan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench

Qinyuan Ye, Harvey Yiyun Fu, Xiang Ren, Robin Jia

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Accepted to EMNLP 2023 Findings. Camera-ready version. Code: https://github.com/INK-USC/predicting-big-bench

Enabling Large Language Models to Generate Text with Citations

Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Comments Accepted by EMNLP 2023. Code and data are available at https://github.com/princeton-nlp/ALCE

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Statistical Knowledge Assessment for Large Language Models

Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Zhifang Sui, Lei Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Accepted by NeurIPS 2023

Evaluation of large language models using an Indian language LGBTI+ lexicon

Aditya Joshi, Shruta Rawat, Alpana Dange

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Selected for publication in the AI Ethics Journal published by the Artificial Intelligence Robotics Ethics Society (AIRES)

WebWISE: Web Interface Control and Sequential Exploration with Large Language Models

Heyi Tao, Sethuraman T, Michal Shlapentokh-Rothman, Derek Hoiem

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Document-Level Machine Translation with Large Language Models

Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, Zhaopeng Tu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang are equal contributors

Zero-Shot Cross-Lingual Summarization via Large Language Models

Jiaan Wang, Yunlong Liang, Fandong Meng, Beiqi Zou, Zhixu Li, Jianfeng Qu, Jie Zhou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Both first authors contributed equally. Technical Report, 12 pages. Accepted to the 4th New Frontiers in Summarization Workshop (NewSumm@EMNLP 2023)

TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge

Ali Maatouk, Fadhel Ayed, Nicola Piovesan, Antonio De Domenico, Merouane Debbah, Zhi-Quan Luo

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries

Yiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu, Munmun De Choudhury, Srijan Kumar

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 7 figures

Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

Erfan Shayegani, Md Abdullah Al Mamun, Yu Fu, Pedram Zaree, Yue Dong, Nael Abu-Ghazaleh

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；分类 cs.CL、cs.LG

Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparison

Shuo Sun, Yuchen Zhang, Jiahuan Yan, Yuze Gao, Donovan Ong, Bin Chen, Jian Su

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Findings of EMNLP 2023

GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Models

Bruno Silva, Leonardo Nunes, Roberto Estevão, Vijay Aski, Ranveer Chandra

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters

Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, Jia Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 14 pages

Journal ref EMNLP 2023

Generalizable Long-Horizon Manipulations with Large Language Models

Haoyu Zhou, Mingyu Ding, Weikun Peng, Masayoshi Tomizuka, Lin Shao, Chuang Gan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?

Johannes Frey, Lars-Peter Meyer, Natanael Arndt, Felix Brei, Kirill Bulert

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments accepted for proceedings of DL4KG Workshop @ ISWC 2023 at ceur-ws.org

Large Language Model Routing with Benchmark Datasets

Tal Shnitzer, Anthony Ou, Mírian Silva, Kate Soule, Yuekai Sun, Justin Solomon, Neil Thompson, Mikhail Yurochkin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 8 figures, 4 tables

Large Language Model Alignment: A Survey

Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, Deyi Xiong

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments 76 pages

An In-depth Survey of Large Language Model-based Artificial Intelligence Agents

Pengyu Zhao, Zijian Jin, Ning Cheng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Text Style Transfer Evaluation Using Large Language Models

Phil Ostheimer, Mayank Nagda, Marius Kloft, Sophie Fellenz

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Considerations for health care institutions training large language models on electronic health records

Weipeng Zhou, Danielle Bitterman, Majid Afshar, Timothy A. Miller

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI