语言大模型 / LLM - arXivDaily 专题

2406.12114 2024-06-19 cs.CL cs.AI cs.LG 85%

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

Hamidreza Rouzegar, Masoud Makrehchi

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Publisher: Association for Computational Linguistics URL: https://aclanthology.org/2024.law-1.10

Journal ref Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII), 2024, pp. 98-111

2402.11485 2024-06-07 cs.CL cs.AI cs.LG 85%

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation

Ikuya Yamada, Ryokan Ri

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL Findings 2024

2211.04325 2024-06-06 cs.LG cs.AI cs.CL cs.CV cs.CY 85%

Will we run out of data? Limits of LLM scaling based on human-generated data

Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, Marius Hobbhahn

专题命中预训练与数据：LLM(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2405.15728 2024-05-27 cs.CV 85%

Disease-informed Adaptation of Vision-Language Models

Jiajin Zhang, Ge Wang, Mannudeep K. Kalra, Pingkun Yan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；prompting(abstract)

Comments Early Accepted by MICCAI 2024

2307.05222 2024-05-09 cs.CV 85%

Emu: Generative Pretraining in Multimodality

Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, Xinlong Wang

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；instruction tuning(abstract)

Comments Accepted to ICLR 2024. Code and Models: https://github.com/baaivision/Emu

2310.02226 2024-04-23 cs.CL cs.AI cs.LG 85%

Think before you speak: Training Language Models With Pause Tokens

Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at ICLR 2024

2404.02145 2024-04-18 cs.CV 85%

Iterated Learning Improves Compositionality in Large Vision-Language Models

Chenhao Zheng, Jieyu Zhang, Aniruddha Kembhavi, Ranjay Krishna

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；language agent(abstract)

Comments CVPR 2024

2404.07612 2024-04-12 cs.CY 85%

Measuring Geographic Diversity of Foundation Models with a Natural Language--based Geo-guessing Experiment on GPT-4

Zilong Liu, Krzysztof Janowicz, Kitty Currier, Meilin Shi

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)

Comments Short paper accepted by AGILE 2024 conference (https://agile-gi.eu/conference-2024)

2403.19063 2024-03-29 cs.IR 85%

Instruction-based Hypergraph Pretraining

Mingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu, Chen Wang, Hao Peng, Philip S. Yu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；prompting(abstract)

Comments Accepted by SIGIR'24

2312.01629 2024-03-28 cs.CV 85%

CLAMP: Contrastive LAnguage Model Prompt-tuning

Piotr Teterwak, Ximeng Sun, Bryan A. Plummer, Kate Saenko, Ser-Nam Lim

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)

2306.00618 2024-03-22 cs.CL cs.AI cs.LG 85%

Effective Structured Prompting by Meta-Learning and Representative Verbalizer

Weisen Jiang, Yu Zhang, James T. Kwok

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICML 2023

2403.09629 2024-03-19 cs.CL cs.AI cs.LG 85%

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.09675 2024-03-18 cs.CV cs.GR 85%

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, Daniel Ritchie

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments See ancillary files for link to supplemental material

2401.15459 2024-03-13 cs.SE 85%

Multi-LLM Collaboration + Data-Centric Innovation = 2x Better Vulnerability Repair

Xin Zhou, Kisub Kim, Bowen Xu, DongGyun Han, David Lo

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted in the ICSE 2024 Research Track with a different title "Out of Sight, Out of Mind: Better Automatic Vulnerability Repair by Broadening Input Ranges and Sources"

2403.00871 2024-03-05 cs.CR cs.AI cs.CL cs.LG 85%

Teach LLMs to Phish: Stealing Private Information from Language Models

Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, Prateek Mittal

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024

2310.06714 2024-02-26 cs.AI cs.CL cs.LG 85%

Exploring Memorization in Fine-tuned Language Models

Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.11777 2024-02-20 cs.CL cs.AI cs.LG 85%

Uncovering Latent Human Wellbeing in Language Model Embeddings

Pedro Freire, ChengCheng Tan, Adam Gleave, Dan Hendrycks, Scott Emmons

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, 5 figures, 1 table

2401.06059 2024-01-12 cs.CL cs.AI cs.LG 85%

Investigating Data Contamination for Pre-training Language Models

Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, Sanmi Koyejo

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 5 figures

2309.17352 2024-01-11 cs.SD eess.AS 85%

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation

Shih-Lun Wu, Xuankai Chang, Gordon Wichern, Jee-weon Jung, François Germain, Jonathan Le Roux, Shinji Watanabe

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments ICASSP 2024 camera-ready paper. Winner of the DCASE 2023 Challenge Task 6A: Automated Audio Captioning (AAC)

2401.03851 2024-01-09 cs.CV q-bio.NC 85%

Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex

Shuxiao Ma, Linyuan Wang, Senbao Hou, Bin Yan

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

2309.03661 2023-12-15 cs.CV 85%

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；prompting(abstract)

Comments 12 pages

2312.07843 2023-12-14 cs.RO 85%

Foundation Models in Robotics: Applications, Challenges, and the Future

Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, Brian Ichter, Danny Driess, Jiajun Wu, Cewu Lu, Mac Schwager

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)

2311.01967 2023-11-06 cs.CL cs.AI cs.LG 85%

The language of prompting: What linguistic properties make a prompt successful?

Alina Leidinger, Robert van Rooij, Ekaterina Shutova

专题命中预训练与数据：prompting(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2023 Findings

2310.13291 2023-10-23 cs.CL cs.AI cs.LG 85%

Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks

Ruixiang Tang, Gord Lueck, Rodolfo Quispe, Huseyin A Inan, Janardhan Kulkarni, Xia Hu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2310.12303 2023-10-20 cs.CL cs.AI cs.LG 85%

Document-Level Language Models for Machine Translation

Frithjof Petrick, Christian Herold, Pavel Petrushkov, Shahram Khadivi, Hermann Ney

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted at WMT 2023

2306.09442 2023-10-12 cs.CL cs.AI cs.LG 85%

Explore, Establish, Exploit: Red Teaming Language Models from Scratch

Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, Dylan Hadfield-Menell

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2306.09085 2023-06-16 cs.CV cs.AI cs.CL cs.LG cs.MM 85%

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Sihan Chen, Xingjian He, Handong Li, Xiaojie Jin, Jiashi Feng, Jing Liu

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2301.13823 2023-06-16 cs.CL cs.AI cs.CV cs.LG 85%

Grounding Language Models to Images for Multimodal Inputs and Outputs

Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in ICML 2023. Project page: https://jykoh.com/fromage

2306.02592 2023-06-06 cs.CL cs.AI cs.LG 85%

Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications

Han Xie, Da Zheng, Jun Ma, Houyu Zhang, Vassilis N. Ioannidis, Xiang Song, Qing Ping, Sheng Wang, Carl Yang, Yi Xu, Belinda Zeng, Trishul Chilimbi

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in the KDD 2023 proceedings as a full paper

2207.06991 2023-04-27 cs.CL cs.AI cs.CV cs.LG 85%

Language Modelling with Pixels

Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, Desmond Elliott

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2023

1. 预训练与数据 12365 篇

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation

Will we run out of data? Limits of LLM scaling based on human-generated data

Disease-informed Adaptation of Vision-Language Models

Emu: Generative Pretraining in Multimodality

Think before you speak: Training Language Models With Pause Tokens

Iterated Learning Improves Compositionality in Large Vision-Language Models

Measuring Geographic Diversity of Foundation Models with a Natural Language--based Geo-guessing Experiment on GPT-4

Instruction-based Hypergraph Pretraining

CLAMP: Contrastive LAnguage Model Prompt-tuning

Effective Structured Prompting by Meta-Learning and Representative Verbalizer

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases

Multi-LLM Collaboration + Data-Centric Innovation = 2x Better Vulnerability Repair

Teach LLMs to Phish: Stealing Private Information from Language Models

Exploring Memorization in Fine-tuned Language Models

Uncovering Latent Human Wellbeing in Language Model Embeddings

Investigating Data Contamination for Pre-training Language Models

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation

Aligned with LLM: a new multi-modal training paradigm for encoding fMRI activity in visual cortex

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Foundation Models in Robotics: Applications, Challenges, and the Future

The language of prompting: What linguistic properties make a prompt successful?

Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks

Document-Level Language Models for Machine Translation

Explore, Establish, Exploit: Red Teaming Language Models from Scratch

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Grounding Language Models to Images for Multimodal Inputs and Outputs

Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications

Language Modelling with Pixels