语言大模型 / LLM

Parallel Corpus Filtering via Pre-trained Language Models

Boliang Zhang, Ajay Nagesh, Kevin Knight

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ACL 2020

Rethinking Exposure Bias In Language Modeling

Yifan Xu, Kening Zhang, Haoyu Dong, Yuezhou Sun, Wenlong Zhao, Zhuowen Tu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

Vishvak Murahari, Dhruv Batra, Devi Parikh, Abhishek Das

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

FlauBERT: Unsupervised Language Model Pre-training for French

Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, Didier Schwab

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to LREC 2020

Masking Orchestration: Multi-task Pretraining for Multi-role Dialogue Representation Learning

Tianyi Wang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Qiong Zhang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 4 figures, AAAI2020

REALM: Retrieval-Augmented Language Model Pre-Training

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Audio-attention discriminative language model for ASR rescoring

Ankur Gandhe, Ariya Rastrow

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 4 pages, 1 figure, Accepted at ICASSP 2020

Unsupervised pretraining transfers well across languages

Morgane Rivière, Armand Joulin, Pierre-Emmanuel Mazaré, Emmanuel Dupoux

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments 6 pages. Accepted at ICASSP 2020. However the 2 pages of supplementary materials will appear only in the arxiv version

Journal ref ICASSP 2020

Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining

Wen-Chin Huang, Tomoki Hayashi, Yi-Chiao Wu, Hirokazu Kameoka, Tomoki Toda

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments Preprint. Work in progress

Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection

David Ifeoluwa Adelani, Haotian Mai, Fuming Fang, Huy H. Nguyen, Junichi Yamagishi, Isao Echizen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments The 34-th International Conference on Advanced Information Networking and Applications (AINA-2020)

A Subword Level Language Model for Bangla Language

Aisha Khatun, Anisur Rahman, Hemayet Ahmed Chowdhury, Md. Saiful Islam, Ayesha Tasnim

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 12 pages, Conference Paper

Pretraining-Based Natural Language Generation for Text Summarization

Haoyu Zhang, Jianjun Xu, Ji Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments 7 pages

Journal ref CoNLL'2019

Robust Navigation with Language Pretraining and Stochastic Sampling

Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 4 figures, EMNLP 2019

Connecting and Comparing Language Model Interpolation Techniques

Ernest Pusateri, Christophe Van Gysel, Rami Botros, Sameer Badaskar, Mirko Hannemann, Youssef Oualil, Ilya Oparin

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection

Sooji Han, Jie Gao, Fabio Ciravegna

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages

Journal ref The 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining

Improved low-resource Somali speech recognition by semi-supervised acoustic and language model training

Astik Biswas, Raghav Menon, Ewald van der Westhuizen, Thomas Niesler

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 5 pages, 6 Tables, 3 figures, 22 references (Accepted at Interspeech 2019)

Analyzing the Structure of Attention in a Transformer Language Model

Jesse Vig, Yonatan Belinkov

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments To appear in ACL BlackboxNLP workshop

Telephonetic: Making Neural Language Models Robust to ASR and Semantic Noise

Chris Larson, Tarek Lahlou, Diana Mingels, Zachary Kulis, Erik Mueller

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Variational Pretraining for Semi-supervised Text Classification

Suchin Gururangan, Tam Dang, Dallas Card, Noah A. Smith

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments ACL 2019

Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains

Navid Rekabsaz, Nikolaos Pappas, James Henderson, Banriskhem K. Khonglah, Srikanth Madikeri

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Pretraining Methods for Dialog Context Representation Learning

Shikib Mehri, Evgeniia Razumovskaia, Tiancheng Zhao, Maxine Eskenazi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2019

Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge

Yu-Ping Ruan, Xiaodan Zhu, Zhen-Hua Ling, Zhan Shi, Quan Liu, Si Wei

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments 7 pages

Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task

Liang Qiu, Yuanyi Ding, Lei He

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition

Shubham Toshniwal, Anjuli Kannan, Chung-Cheng Chiu, Yonghui Wu, Tara N Sainath, Karen Livescu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in SLT 2018

Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling

Jaejin Cho, Murali Karthick Baskar, Ruizhi Li, Matthew Wiesner, Sri Harish Mallidi, Nelson Yalta, Martin Karafiat, Shinji Watanabe, Takaaki Hori

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Syntax-Aware Language Modeling with Recurrent Neural Networks

Duncan Blythe, Alan Akbik, Roland Vollgraf

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations

Xiaoqin Zhang, Huimin Ma

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Added acknowledgements, modified references. 7 pages, 4 figures

Patterns versus Characters in Subword-aware Neural Language Modeling

Rustem Takhanov, Zhenisbek Assylbekov

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 10 pages

A Study on Neural Network Language Modeling

Dengliang Shi

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 20 pages, 6 figures

A Neural Network Approach for Mixing Language Models

Youssef Oualil, Dietrich Klakow

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Published at IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2017. arXiv admin note: text overlap with arXiv:1703.08068

Journal ref IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, 2017, pp. 5710-5714