语言大模型 / LLM - arXivDaily 专题

2408.07091 2024-08-22 cs.LG cs.AI cs.CL 85%

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

Wenbin Hu, Huihao Jing, Qi Hu, Haoran Li, Yangqiu Song

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2407.20371 2024-08-22 cs.CY cs.AI cs.CL cs.LG 85%

Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

Kyra Wilson, Aylin Caliskan

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society; code available at https://github.com/kyrawilson/Resume-Screening-Bias

2308.07922 2024-08-20 cs.CL cs.AI cs.LG 85%

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models

Jie Huang, Wei Ping, Peng Xu, Mohammad Shoeybi, Kevin Chen-Chuan Chang, Bryan Catanzaro

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments COLM 2024

2407.01490 2024-07-22 cs.CL cs.AI cs.LG 85%

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives

Luísa Shimabucoro, Sebastian Ruder, Julia Kreutzer, Marzieh Fadaee, Sara Hooker

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2407.12854 2024-07-19 cs.CL cs.AI cs.IR cs.LG 85%

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore

Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, Pang Wei Koh

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2406.13972 2024-07-09 cs.SE 85%

CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors

Boyang Yang, Haoye Tian, Weiguo Pian, Haoran Yu, Haitao Wang, Jacques Klein, Tegawendé F. Bissyandé, Shunfu Jin

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

2407.04489 2024-07-08 cs.CV 85%

Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model

Duy M. H. Nguyen, An T. Le, Trung Q. Nguyen, Nghiem T. Diep, Tai Nguyen, Duy Duong-Tran, Jan Peters, Li Shen, Mathias Niepert, Daniel Sonntag

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Version 1

2406.12114 2024-06-19 cs.CL cs.AI cs.LG 85%

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

Hamidreza Rouzegar, Masoud Makrehchi

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Publisher: Association for Computational Linguistics URL: https://aclanthology.org/2024.law-1.10

Journal ref Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII), 2024, pp. 98-111

2402.11485 2024-06-07 cs.CL cs.AI cs.LG 85%

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation

Ikuya Yamada, Ryokan Ri

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL Findings 2024

2211.04325 2024-06-06 cs.LG cs.AI cs.CL cs.CV cs.CY 85%

Will we run out of data? Limits of LLM scaling based on human-generated data

Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, Marius Hobbhahn

专题命中预训练与数据：LLM(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2405.15728 2024-05-27 cs.CV 85%

Disease-informed Adaptation of Vision-Language Models

Jiajin Zhang, Ge Wang, Mannudeep K. Kalra, Pingkun Yan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；prompting(abstract)

Comments Early Accepted by MICCAI 2024

2307.05222 2024-05-09 cs.CV 85%

Emu: Generative Pretraining in Multimodality

Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, Xinlong Wang

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；instruction tuning(abstract)

Comments Accepted to ICLR 2024. Code and Models: https://github.com/baaivision/Emu

2310.02226 2024-04-23 cs.CL cs.AI cs.LG 85%

Think before you speak: Training Language Models With Pause Tokens

Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at ICLR 2024

2404.02145 2024-04-18 cs.CV 85%

Iterated Learning Improves Compositionality in Large Vision-Language Models

Chenhao Zheng, Jieyu Zhang, Aniruddha Kembhavi, Ranjay Krishna

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；language agent(abstract)

Comments CVPR 2024

2404.07612 2024-04-12 cs.CY 85%

Measuring Geographic Diversity of Foundation Models with a Natural Language--based Geo-guessing Experiment on GPT-4

Zilong Liu, Krzysztof Janowicz, Kitty Currier, Meilin Shi

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)

Comments Short paper accepted by AGILE 2024 conference (https://agile-gi.eu/conference-2024)

2403.19063 2024-03-29 cs.IR 85%

Instruction-based Hypergraph Pretraining

Mingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu, Chen Wang, Hao Peng, Philip S. Yu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；prompting(abstract)

Comments Accepted by SIGIR'24

2312.01629 2024-03-28 cs.CV 85%

CLAMP: Contrastive LAnguage Model Prompt-tuning

Piotr Teterwak, Ximeng Sun, Bryan A. Plummer, Kate Saenko, Ser-Nam Lim

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)

2306.00618 2024-03-22 cs.CL cs.AI cs.LG 85%

Effective Structured Prompting by Meta-Learning and Representative Verbalizer

Weisen Jiang, Yu Zhang, James T. Kwok

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICML 2023

2403.09629 2024-03-19 cs.CL cs.AI cs.LG 85%

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.09675 2024-03-18 cs.CV cs.GR 85%

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, Daniel Ritchie

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments See ancillary files for link to supplemental material

2401.15459 2024-03-13 cs.SE 85%

Multi-LLM Collaboration + Data-Centric Innovation = 2x Better Vulnerability Repair

Xin Zhou, Kisub Kim, Bowen Xu, DongGyun Han, David Lo

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted in the ICSE 2024 Research Track with a different title "Out of Sight, Out of Mind: Better Automatic Vulnerability Repair by Broadening Input Ranges and Sources"

2403.00871 2024-03-05 cs.CR cs.AI cs.CL cs.LG 85%

Teach LLMs to Phish: Stealing Private Information from Language Models

Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, Prateek Mittal

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024

2310.06714 2024-02-26 cs.AI cs.CL cs.LG 85%

Exploring Memorization in Fine-tuned Language Models

Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.11777 2024-02-20 cs.CL cs.AI cs.LG 85%

Uncovering Latent Human Wellbeing in Language Model Embeddings

Pedro Freire, ChengCheng Tan, Adam Gleave, Dan Hendrycks, Scott Emmons

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, 5 figures, 1 table

2401.06059 2024-01-12 cs.CL cs.AI cs.LG 85%

Investigating Data Contamination for Pre-training Language Models

Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, Sanmi Koyejo

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 5 figures

2309.17352 2024-01-11 cs.SD eess.AS 85%

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation

Shih-Lun Wu, Xuankai Chang, Gordon Wichern, Jee-weon Jung, François Germain, Jonathan Le Roux, Shinji Watanabe

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments ICASSP 2024 camera-ready paper. Winner of the DCASE 2023 Challenge Task 6A: Automated Audio Captioning (AAC)

2401.03851 2024-01-09 cs.CV q-bio.NC 85%

Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex

Shuxiao Ma, Linyuan Wang, Senbao Hou, Bin Yan

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

2309.03661 2023-12-15 cs.CV 85%

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；prompting(abstract)

Comments 12 pages

2312.07843 2023-12-14 cs.RO 85%

Foundation Models in Robotics: Applications, Challenges, and the Future

Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, Brian Ichter, Danny Driess, Jiajun Wu, Cewu Lu, Mac Schwager

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)

2311.01967 2023-11-06 cs.CL cs.AI cs.LG 85%

The language of prompting: What linguistic properties make a prompt successful?

Alina Leidinger, Robert van Rooij, Ekaterina Shutova

专题命中预训练与数据：prompting(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2023 Findings

1. 预训练与数据 12393 篇

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore

CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors

Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation

Will we run out of data? Limits of LLM scaling based on human-generated data

Disease-informed Adaptation of Vision-Language Models

Emu: Generative Pretraining in Multimodality

Think before you speak: Training Language Models With Pause Tokens

Iterated Learning Improves Compositionality in Large Vision-Language Models

Measuring Geographic Diversity of Foundation Models with a Natural Language--based Geo-guessing Experiment on GPT-4

Instruction-based Hypergraph Pretraining

CLAMP: Contrastive LAnguage Model Prompt-tuning

Effective Structured Prompting by Meta-Learning and Representative Verbalizer

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Multi-LLM Collaboration + Data-Centric Innovation = 2x Better Vulnerability Repair

Teach LLMs to Phish: Stealing Private Information from Language Models

Exploring Memorization in Fine-tuned Language Models

Uncovering Latent Human Wellbeing in Language Model Embeddings

Investigating Data Contamination for Pre-training Language Models

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation

Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Foundation Models in Robotics: Applications, Challenges, and the Future

The language of prompting: What linguistic properties make a prompt successful?