语言大模型 / LLM - arXivDaily 专题

2303.03922 2023-03-08 cs.LG cs.AI cs.CL 82%

Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer

Wen Zhang, Yushan Zhu, Mingyang Chen, Yuxia Geng, Yufeng Huang, Yajing Xu, Wenting Song, Huajun Chen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work accepted by WWW2023

2302.01842 2023-02-06 cs.AI cs.CL cs.IR cs.LG 82%

A Case Study for Compliance as Code with Graphs and Language Models: Public release of the Regulatory Knowledge Graph

Vladimir Ershov

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 7 pages on RegKG step 1 in collaboration with ADGM

2301.07236 2023-01-19 cs.CV 82%

Effective End-to-End Vision Language Pretraining with Semantic Visual Loss

Xiaofeng Yang, Fayao Liu, Guosheng Lin

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)

2301.03728 2023-01-11 cs.CL cs.AI cs.LG 82%

Scaling Laws for Generative Mixed-Modal Language Models

Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, Luke Zettlemoyer

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2212.03191 2022-12-08 cs.CV 82%

InternVideo: General Video Foundation Models via Generative and Discriminative Learning

Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, Sen Xing, Guo Chen, Junting Pan, Jiashuo Yu, Yali Wang, Limin Wang, Yu Qiao

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)

Comments technical report

2211.12030 2022-11-23 cs.CV 82%

Knowledge Prompting for Few-shot Action Recognition

Yuheng Shi, Xinxiao Wu, Hanxi Lin

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)

2201.11147 2022-11-02 q-bio.BM cs.AI cs.CL cs.IR cs.LG 82%

OntoProtein: Protein Pretraining With Gene Ontology Embedding

Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Jiazhang Lian, Qiang Zhang, Huajun Chen

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ICLR 2022

2210.15138 2022-10-28 cs.CV 82%

Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models

Chaofan Ma, Yuhuan Yang, Yanfeng Wang, Ya Zhang, Weidi Xie

专题命中预训练与数据：language model(title,abstract)；foundation model(abstract)

Comments BMVC 2022 Oral

2210.06722 2022-10-14 cs.LG cs.AI cs.CL 82%

Few-shot Relational Reasoning via Connection Subgraph Pretraining

Qian Huang, Hongyu Ren, Jure Leskovec

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2022. Open source implementation at https://github.com/snap-stanford/csr

2210.04726 2022-10-11 cs.CL cs.AI cs.LG 82%

Knowledge Prompts: Injecting World Knowledge into Language Models through Soft Prompts

Cicero Nogueira dos Santos, Zhe Dong, Daniel Cer, John Nham, Siamak Shakeri, Jianmo Ni, Yun-hsuan Sung

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2207.01622 2022-08-04 cs.CV 82%

Egocentric Video-Language Pretraining @ Ego4D Challenge 2022

Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)

Comments Preprint. 4 pages, 2 figures, 5 tables. Code: https://github.com/showlab/EgoVLP. The Ego4D challenge technical report of EgoVLP arXiv:2206.01670. See EPIC challenge technical report arXiv:2207.01334 for overlap

2207.01334 2022-08-04 cs.CV 82%

Egocentric Video-Language Pretraining @ EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022

Kevin Qinghong Lin, Alex Jinpeng Wang, Rui Yan, Eric Zhongcong Xu, Rongcheng Tu, Yanru Zhu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Wei Liu, Mike Zheng Shou

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)

Comments To appeared in CVPRW22. 5 pages, 2 figures, 2 tables. Code: https://github.com/showlab/EgoVLP. The EPIC challenge technical report of EgoVLP arXiv:2206.01670. See Ego4D challenge technical report arXiv:2207.01622

2208.00475 2022-08-02 cs.CV 82%

Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics

Xiaoyuan Guo, Jiali Duan, C. -C. Jay Kuo, Judy Wawira Gichoya, Imon Banerjee

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)

Comments 7 pages, 4 figures, ICPR2022. arXiv admin note: text overlap with arXiv:2203.00048

2207.11280 2022-07-26 cs.LG cs.AI cs.CL cs.PL cs.SE 82%

PanGu-Coder: Program Synthesis with Function-Level Language Modeling

Fenia Christopoulou, Gerasimos Lampouras, Milan Gritta, Guchun Zhang, Yinpeng Guo, Zhongqi Li, Qi Zhang, Meng Xiao, Bo Shen, Lin Li, Hao Yu, Li Yan, Pingyi Zhou, Xin Wang, Yuchi Ma, Ignacio Iacobacci, Yasheng Wang, Guangtai Liang, Jiansheng Wei, Xin Jiang, Qianxiang Wang, Qun Liu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages

2206.07808 2022-06-17 cs.CL cs.AI cs.LG 82%

Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems

Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas, Davide Bernardi, Abhishek Bhagia, Claudio Delli Bovi, Jin Cao, Rakesh Chada, Amit Chauhan, Luoxin Chen, Anurag Dwarakanath, Satyam Dwivedi, Turan Gojayev, Karthik Gopalakrishnan, Thomas Gueudre, Dilek Hakkani-Tur, Wael Hamza, Jonathan Hueser, Kevin Martin Jose, Haidar Khan, Beiye Liu, Jianhua Lu, Alessandro Manzotti, Pradeep Natarajan, Karolina Owczarzak, Gokmen Oz, Enrico Palumbo, Charith Peris, Chandana Satya Prakash, Stephen Rawls, Andy Rosenbaum, Anjali Shenoy, Saleh Soltan, Mukund Harakere Sridhar, Liz Tan, Fabian Triefenbach, Pan Wei, Haiyang Yu, Shuai Zheng, Gokhan Tur, Prem Natarajan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments KDD 2022

Journal ref Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '22), August 14-18, 2022, Washington, DC, USA

2206.03281 2022-06-08 cs.IR 82%

Unsupervised Context Aware Sentence Representation Pretraining for Multi-lingual Dense Retrieval

Ning Wu, Yaobo Liang, Houxing Ren, Linjun Shou, Nan Duan, Ming Gong, Daxin Jiang

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)

2201.07449 2022-05-20 cs.CL cs.AI cs.LG 82%

TourBERT: A pretrained language model for the tourism industry

Veronika Arefieva, Roman Egger

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Identified a mistake in our calculations. Will fix the problem within the next weeks and resubmit

2205.05368 2022-05-12 cs.CL cs.AI cs.HC cs.LG 82%

Pre-trained Language Models as Re-Annotators

Chang Shu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Thesis of Master of Science by Research (M.Res) in Linguistics with Distinction; University of Edinburgh, 2022; 107 pages

2109.06096 2022-04-07 cs.CL cs.AI cs.LG 82%

The Grammar-Learning Trajectories of Neural Language Models

Leshem Choshen, Guy Hacohen, Daphna Weinshall, Omri Abend

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL camera-ready

2110.06490 2022-03-22 cs.CL cs.AI cs.LG 82%

Dict-BERT: Enhancing Language Model Pre-training with Dictionary

Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2022 (Findings)

2203.07511 2022-03-16 cs.CL cs.AI cs.CY cs.LG 82%

Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations

Robert Wolfe, Aylin Caliskan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in ACL 2022

2203.03072 2022-03-08 cs.CL cs.AI cs.LG 82%

Leashing the Inner Demons: Self-Detoxification for Language Models

Canwen Xu, Zexue He, Zhankui He, Julian McAuley

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments AAAI 2022

2202.03286 2022-02-08 cs.CL cs.AI cs.CR cs.LG 82%

Red Teaming Language Models with Language Models

Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, Geoffrey Irving

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2201.00971 2022-01-05 cs.LG cs.AI cs.CL 82%

Submix: Practical Private Prediction for Large-Scale Language Models

Antonio Ginart, Laurens van der Maaten, James Zou, Chuan Guo

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2111.14745 2021-11-30 cs.CV 82%

A Simple Long-Tailed Recognition Baseline via Vision-Language Model

Teli Ma, Shijie Geng, Mengmeng Wang, Jing Shao, Jiasen Lu, Hongsheng Li, Peng Gao, Yu Qiao

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)

2110.09930 2021-10-20 eess.AS cs.AI cs.CL cs.LG cs.SD 82%

Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning

Yi-Chen Chen, Shu-wen Yang, Cheng-Kuang Lee, Simon See, Hung-yi Lee

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2110.00672 2021-10-05 cs.CY cs.AI cs.CL cs.LG 82%

Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models

Robert Wolfe, Aylin Caliskan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 15 pages, 3 figures, 8 tables

Journal ref Empirical Methods in Natural Language Processing 2021

2108.09105 2021-08-23 cs.CV cs.AI cs.CL cs.HC cs.LG 82%

Airbert: In-domain Pretraining for Vision-and-Language Navigation

Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, Cordelia Schmid

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published on ICCV 2021. Webpage is at https://airbert-vln.github.io/ linking to our dataset, codes and models

2106.01703 2021-06-04 cs.CL cs.AI cs.LG 82%

Fingerprinting Fine-tuned Language Models in the Wild

Nirav Diwan, Tanmoy Chakravorty, Zubair Shafiq

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2103.10518 2021-03-22 cs.CL cs.AI cs.LG 82%

Pretraining the Noisy Channel Model for Task-Oriented Dialogue

Qi Liu, Lei Yu, Laura Rimell, Phil Blunsom

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to TACL, pre MIT Press publication version

1. 预训练与数据 12393 篇

Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer

A Case Study for Compliance as Code with Graphs and Language Models: Public release of the Regulatory Knowledge Graph

Effective End-to-End Vision Language Pretraining with Semantic Visual Loss

Scaling Laws for Generative Mixed-Modal Language Models

InternVideo: General Video Foundation Models via Generative and Discriminative Learning

Knowledge Prompting for Few-shot Action Recognition

OntoProtein: Protein Pretraining With Gene Ontology Embedding

Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models

Few-shot Relational Reasoning via Connection Subgraph Pretraining

Knowledge Prompts: Injecting World Knowledge into Language Models through Soft Prompts

Egocentric Video-Language Pretraining @ Ego4D Challenge 2022

Egocentric Video-Language Pretraining @ EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022

Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics

PanGu-Coder: Program Synthesis with Function-Level Language Modeling

Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems

Unsupervised Context Aware Sentence Representation Pretraining for Multi-lingual Dense Retrieval

TourBERT: A pretrained language model for the tourism industry

Pre-trained Language Models as Re-Annotators

The Grammar-Learning Trajectories of Neural Language Models

Dict-BERT: Enhancing Language Model Pre-training with Dictionary

Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations

Leashing the Inner Demons: Self-Detoxification for Language Models

Red Teaming Language Models with Language Models

Submix: Practical Private Prediction for Large-Scale Language Models

A Simple Long-Tailed Recognition Baseline via Vision-Language Model

Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning

Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models

Airbert: In-domain Pretraining for Vision-and-Language Navigation

Fingerprinting Fine-tuned Language Models in the Wild

Pretraining the Noisy Channel Model for Task-Oriented Dialogue