语言大模型 / LLM - arXivDaily 专题

2004.02451 2020-06-09 cs.CL 79%

An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Models

Hiroshi Noji, Hiroya Takamura

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2020 camera ready (long paper); code is available at https://github.com/aistairc/lm_syntax_negative

2006.02144 2020-06-04 cs.CL 79%

Transfer Learning for British Sign Language Modelling

Boris Mocialov, Graham Turner, Helen Hastie

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 10 pages, 3 figures

Journal ref Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects, VarDial (2018)

2005.00987 2020-06-02 cs.CL 79%

Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction

Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, Kentaro Inui

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted as a short paper to the 58th Annual Conference of the Association for Computational Linguistics (ACL-2020)

Journal ref Association for Computational Linguistics (ACL-2020)

1910.10387 2020-05-25 cs.CL eess.AS 79%

Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks

Xingchen Song, Guangsen Wang, Zhiyong Wu, Yiheng Huang, Dan Su, Dong Yu, Helen Meng

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments \c{opyright} 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

2005.05477 2020-05-14 cs.CL 79%

Neural Polysynthetic Language Modelling

Lane Schwartz, Francis Tyers, Lori Levin, Christo Kirov, Patrick Littell, Chi-kiu Lo, Emily Prud'hommeaux, Hyunji Hayley Park, Kenneth Steimel, Rebecca Knowles, Jeffrey Micher, Lonny Strunk, Han Liu, Coleman Haley, Katherine J. Zhang, Robbie Jimmerson, Vasilisa Andriyanets, Aldrian Obaja Muis, Naoki Otani, Jong Hyuk Park, Zhisong Zhang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2005.00165 2020-05-08 cs.CL 79%

Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment

Forrest Davis, Marten van Schijndel

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Proceedings of 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020); v3 updated references and added additional corpus stats in discussion

2005.01190 2020-05-05 cs.CL 79%

Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models

Kaiji Lu, Piotr Mardziel, Klas Leino, Matt Fedrikson, Anupam Datta

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2020

2004.13214 2020-04-29 cs.SE cs.LG 79%

SCELMo: Source Code Embeddings from Language Models

Rafael - Michael Karampatsis, Charles Sutton

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments 12 pages

2004.08744 2020-04-21 cs.CV cs.CL 79%

Are we pretraining it right? Digging deeper into visio-linguistic pretraining

Amanpreet Singh, Vedanuj Goswami, Devi Parikh

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 23 pages, 6 figures. First two authors contributed equally. More info at https://github.com/facebookresearch/pythia

2004.05910 2020-04-14 cs.LG stat.ML 79%

Training few-shot classification via the perspective of minibatch and pretraining

Meiyu Huang, Xueshuang Xiang, Yao Xu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments arXiv admin note: text overlap with arXiv:1803.00676 by other authors

2003.14323 2020-04-01 cs.CV cs.LG 79%

How Useful is Self-Supervised Pretraining for Visual Tasks?

Alejandro Newell, Jia Deng

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments To appear in CVPR 2020

2003.01355 2020-03-06 cs.CL 79%

CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model

Liang Xu, Xuanwei Zhang, Qianqian Dong

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 8 pages, 9 tables

2002.12804 2020-03-02 cs.CL 79%

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 11 pages

2002.11268 2020-03-02 eess.AS cs.CL cs.SD 79%

A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognition

Erik McDermott, Hasim Sak, Ehsan Variani

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 8 pages, 4 figures, presented at 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU 2019)

1911.00172 2020-02-18 cs.CL 79%

Generalization through Memorization: Nearest Neighbor Language Models

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ICLR 2020

1910.10762 2020-02-11 cs.CL eess.AS 79%

Analyzing ASR pretraining for low-resource speech-to-text translation

Mihaela C. Stoian, Sameer Bansal, Sharon Goldwater

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted at ICASSP 2020

2001.08361 2020-01-24 cs.LG stat.ML 79%

Scaling Laws for Neural Language Models

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments 19 pages, 15 figures

1909.11299 2020-01-24 cs.LG stat.ML 79%

Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models

Cheolhyoung Lee, Kyunghyun Cho, Wanmo Kang

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments Published as a conference paper at ICLR 2020

2001.05139 2020-01-16 cs.CL 79%

A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation

Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, Minlie Huang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accept at Transactions of the Association for Computational Linguistics 2020

1912.10308 2019-12-24 cs.CV cs.CL 79%

Candidate Fusion: Integrating Language Modelling into a Sequence-to-Sequence Handwritten Word Recognition Architecture

Lei Kang, Pau Riba, Mauricio Villegas, Alicia Fornés, Marçal Rusiñol

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

1912.07095 2019-12-17 cs.CL 79%

Robust Named Entity Recognition with Truecasing Pretraining

Stephen Mayhew, Nitish Gupta, Dan Roth

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted to AAAI 2020

1912.05421 2019-12-12 cs.LG stat.ML 79%

Just Add Functions: A Neural-Symbolic Language Model

David Demeter, Doug Downey

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments Preprint of paper accepted for AAAI-2020

1911.11489 2019-11-27 cs.CL 79%

Relevance-Promoting Language Model for Short-Text Conversation

Xin Li, Piji Li, Wei Bi, Xiaojiang Liu, Wai Lam

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments AAAI 2020

1911.04571 2019-11-13 cs.CL cs.SD eess.AS 79%

Long-span language modeling for speech recognition

Sarangarajan Parthasarathy, William Gale, Xie Chen, George Polovets, Shuangyu Chang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

1911.03937 2019-11-12 cs.CL 79%

Language Model-Driven Unsupervised Neural Machine Translation

Wei Zhang, Youyuan Lin, Ruoran Ren, Xiaodong Wang, Zhenshuang Liang, Zhen Huang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 11 pages, 3 figures, 7 tables

1911.03597 2019-11-12 cs.CL 79%

Zero-Shot Paraphrase Generation with Multilingual Language Models

Yinpeng Guo, Yi Liao, Xin Jiang, Qing Zhang, Yibo Zhang, Qun Liu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

1911.01071 2019-11-05 cs.LG cs.CV stat.ML 79%

Supervised level-wise pretraining for recurrent neural network initialization in multi-class classification

Dino Ienco, Roberto Interdonato, Raffaele Gaetano

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

1909.11861 2019-10-08 cs.CL 79%

Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs

Yuxian Meng, Xiangyuan Ren, Zijun Sun, Xiaoya Li, Arianna Yuan, Fei Wu, Jiwei Li

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

1909.10599 2019-09-25 cs.CL 79%

Multi-stage Pretraining for Abstractive Summarization

Sebastian Goodman, Zhenzhong Lan, Radu Soricut

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

1909.05858 2019-09-24 cs.CL 79%

CTRL: A Conditional Transformer Language Model for Controllable Generation

Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, Richard Socher

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

1. 预训练与数据 12393 篇

An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Models

Transfer Learning for British Sign Language Modelling

Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction

Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks

Neural Polysynthetic Language Modelling

Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment

Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models

SCELMo: Source Code Embeddings from Language Models

Are we pretraining it right? Digging deeper into visio-linguistic pretraining

Training few-shot classification via the perspective of minibatch and pretraining

How Useful is Self-Supervised Pretraining for Visual Tasks?

CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognition

Generalization through Memorization: Nearest Neighbor Language Models

Analyzing ASR pretraining for low-resource speech-to-text translation

Scaling Laws for Neural Language Models

Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models

A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation

Candidate Fusion: Integrating Language Modelling into a Sequence-to-Sequence Handwritten Word Recognition Architecture

Robust Named Entity Recognition with Truecasing Pretraining

Just Add Functions: A Neural-Symbolic Language Model

Relevance-Promoting Language Model for Short-Text Conversation

Long-span language modeling for speech recognition

Language Model-Driven Unsupervised Neural Machine Translation

Zero-Shot Paraphrase Generation with Multilingual Language Models

Supervised level-wise pretraining for recurrent neural network initialization in multi-class classification

Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs

Multi-stage Pretraining for Abstractive Summarization

CTRL: A Conditional Transformer Language Model for Controllable Generation