语言大模型 / LLM - arXivDaily 专题

2012.12543 2020-12-25 cs.CL cs.SD eess.AS 79%

Code Switching Language Model Using Monolingual Training Data

Asad Ullah, Tauseef Ahmed

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments submitted to ICASSP2021

2012.11995 2020-12-23 cs.CL 79%

Pre-Training a Language Model Without Human Language

Cheng-Han Chiang, Hung-yi Lee

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 9 pages, work in progress

2012.07331 2020-12-15 eess.AS cs.CL cs.SD 79%

Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval

Yuma Koizumi, Yasunori Ohishi, Daisuke Niizumi, Daiki Takeuchi, Masahiro Yasuda

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Submitted to ICASSP 2021

2012.03084 2020-12-08 q-bio.BM cs.CL 79%

Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks

Modestas Filipavicius, Matteo Manica, Joris Cadow, Maria Rodriguez Martinez

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 20 pages, 12 figures, accepted to Machine Learning for Structural Biology (MLSB) workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS)

2012.00413 2020-12-02 cs.CL 79%

CPM: A Large-scale Generative Chinese Pre-trained Language Model

Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2011.08626 2020-11-20 cs.CL 79%

Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining

Zijun Sun, Chun Fan, Xiaofei Sun, Yuxian Meng, Fei Wu, Jiwei Li

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2011.04839 2020-11-11 cs.SD cs.CL 79%

Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis

Erica Cooper, Xin Wang, Yi Zhao, Yusuke Yasuda, Junichi Yamagishi

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Technical report

2007.13002 2020-10-30 eess.AS cs.CL cs.SD 79%

Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling

Siyuan Feng, Odette Scharenborg

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 5 pages, 3 figures. Accepted for publication in INTERSPEECH 2020, Shanghai, China

2010.13870 2020-10-28 cs.CL 79%

Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Charles Yu, Ryan Sie, Nico Tedeschi, Leon Bergen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2020

2010.11639 2020-10-23 cs.CL 79%

Towards Fully Bilingual Deep Language Modeling

Li-Hsin Chang, Sampo Pyysalo, Jenna Kanerva, Filip Ginter

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2010.11553 2020-10-23 cs.CL 79%

Incorporating Stylistic Lexical Preferences in Generative Language Models

Hrituraj Singh, Gaurav Verma, Balaji Vasan Srinivasan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To Appear in Findings of EMNLP 2020

2010.06804 2020-10-15 cs.CL 79%

Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion

Ankur Goswami, Akshata Bhat, Hadar Ohana, Theodoros Rekatsinas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 14 pages, 5 figures, Accepted to Findings of EMNLP 2020

2006.10413 2020-10-13 cs.CL 79%

Are Pretrained Language Models Symbolic Reasoners Over Knowledge?

Nora Kassner, Benno Krojer, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to CoNLL 2020

2010.01150 2020-10-06 cs.CL 79%

Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media

Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2020

2010.01054 2020-10-05 cs.CL 79%

Unsupervised Text Style Transfer with Padded Masked Language Models

Eric Malmi, Aliaksei Severyn, Sascha Rothe

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2020

2009.10031 2020-09-22 cs.LG cs.CR stat.ML 79%

Training Production Language Models without Memorizing User Data

Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H. Brendan McMahan, Françoise Beaufays

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2009.09223 2020-09-22 cs.CL 79%

BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition

Usman Naseem, Matloob Khushi, Vinay Reddy, Sakthivel Rajendran, Imran Razzak, Jinman Kim

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 7 pages

2004.07159 2020-09-22 cs.CL 79%

PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation

Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP 2020

2004.03012 2020-09-17 cs.CL 79%

"You are grounded!": Latent Name Artifacts in Pre-trained Language Models

Vered Shwartz, Rachel Rudinger, Oyvind Tafjord

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2020

2004.00584 2020-09-04 cs.DB cs.CL 79%

Deep Entity Matching with Pre-Trained Language Models

Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, Wang-Chiew Tan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in VLDB 2021

2005.04828 2020-08-19 cs.LG stat.ML 79%

Pretraining Federated Text Models for Next Word Prediction

Joel Stremmel, Arjun Singh

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2008.03088 2020-08-10 eess.AS cs.CL cs.SD 79%

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

Wen-Chin Huang, Tomoki Hayashi, Yi-Chiao Wu, Hirokazu Kameoka, Tomoki Toda

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Preprint. Under review

2008.00401 2020-08-04 cs.CL 79%

Multilingual Translation with Extensible Multilingual Pretraining and Finetuning

Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, Angela Fan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 10 pages (main) + 5 pages (appendices). 9 tables and 2 figures

2005.14165 2020-07-24 cs.CL 79%

Language Models are Few-Shot Learners

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 40+32 pages

2007.07423 2020-07-23 cs.CV cs.LG 79%

Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representations

Hong-Yu Zhou, Shuang Yu, Cheng Bian, Yifan Hu, Kai Ma, Yefeng Zheng

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments MICCAI 2020 early accept; Code and pretrained models available at MICCAI2020" target="_blank" rel="noopener">http://github.com/funnyzhou/C2L_MICCAI2020

2007.06162 2020-07-14 cs.CL 79%

Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods

Ning Miao, Yuxuan Song, Hao Zhou, Lei Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2020

2007.04234 2020-07-09 cs.CV cs.LG stat.ML 79%

Transfer Learning or Self-supervised Learning? A Tale of Two Pretraining Paradigms

Xingyi Yang, Xuehai He, Yuxiao Liang, Yue Yang, Shanghang Zhang, Pengtao Xie

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2007.01528 2020-07-06 cs.CL 79%

On-The-Fly Information Retrieval Augmentation for Language Models

Hai Wang, David McAllester

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2020 NUSE Workshop

2006.16743 2020-07-01 cs.HC cs.CL cs.PL cs.SE 79%

Learning to Format Coq Code Using Language Models

Pengyu Nie, Karl Palmskog, Junyi Jessy Li, Milos Gligoric

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted in the Coq Workshop 2020

2006.04229 2020-06-11 cs.CL 79%

Pre-training Polish Transformer-based Language Models at Scale

Sławomir Dadas, Michał Perełkiewicz, Rafał Poświata

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

1. 预训练与数据 12393 篇

Code Switching Language Model Using Monolingual Training Data

Pre-Training a Language Model Without Human Language

Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval

Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks

CPM: A Large-scale Generative Chinese Pre-trained Language Model

Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining

Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis

Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling

Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Towards Fully Bilingual Deep Language Modeling

Incorporating Stylistic Lexical Preferences in Generative Language Models

Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion

Are Pretrained Language Models Symbolic Reasoners Over Knowledge?

Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media

Unsupervised Text Style Transfer with Padded Masked Language Models

Training Production Language Models without Memorizing User Data

BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition

PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation

"You are grounded!": Latent Name Artifacts in Pre-trained Language Models

Deep Entity Matching with Pre-Trained Language Models

Pretraining Federated Text Models for Next Word Prediction

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

Multilingual Translation with Extensible Multilingual Pretraining and Finetuning

Language Models are Few-Shot Learners

Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representations

Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods

Transfer Learning or Self-supervised Learning? A Tale of Two Pretraining Paradigms

On-The-Fly Information Retrieval Augmentation for Language Models

Learning to Format Coq Code Using Language Models

Pre-training Polish Transformer-based Language Models at Scale