语言大模型 / LLM - arXivDaily 专题

2105.07144 2021-06-11 cs.CL 79%

A Cognitive Regularizer for Language Modeling

Jason Wei, Clara Meister, Ryan Cotterell

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2021 Camera-ready (fixed ordering of affiliation emojis)

2106.04799 2021-06-10 cs.LG 79%

Pretraining Representations for Data-Efficient Reinforcement Learning

Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, Devon Hjelm, Philip Bachman, Aaron Courville

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2106.04121 2021-06-09 cs.CV cs.LG 79%

Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

Bowen Shi, Xiaopeng Zhang, Haohang Xu, Wenrui Dai, Junni Zou, Hongkai Xiong, Qi Tian

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2106.04098 2021-06-09 cs.CL 79%

Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

Hongliang Dai, Yangqiu Song, Haixun Wang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2021

2104.04552 2021-06-08 cs.CL cs.SD eess.AS 79%

Lookup-Table Recurrent Language Models for Long Tail Speech Recognition

W. Ronny Huang, Tara N. Sainath, Cal Peyser, Shankar Kumar, David Rybach, Trevor Strohman

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Presented as conference paper at Interspeech 2021

2106.01091 2021-06-03 cs.CL 79%

belabBERT: a Dutch RoBERTa-based language model applied to psychiatric classification

Joppe Wouts, Janna de Boer, Alban Voppel, Sanne Brederoo, Sander van Splunter, Iris Sommer

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments arXiv admin note: substantial text overlap with arXiv:2008.01543

2106.00737 2021-06-03 cs.CL 79%

Implicit Representations of Meaning in Neural Language Models

Belinda Z. Li, Maxwell Nye, Jacob Andreas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 15 pages, 6 figures; accepted to ACL 2021

2101.00438 2021-06-03 cs.CL 79%

Few-Shot Question Answering by Pretraining Span Selection

Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2021

2011.15124 2021-06-01 cs.CL cs.CV 79%

Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs

Emanuele Bugliarello, Ryan Cotterell, Naoaki Okazaki, Desmond Elliott

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments To appear in TACL 2021

2105.12810 2021-05-28 cs.CV cs.LG 79%

ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans

Hasib Zunair, Aimon Rahman, Nabeel Mohammed

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Under review at CLEF 2021. 10 pages

2011.04393 2021-05-26 cs.CL 79%

Positional Artefacts Propagate Through Masked Language Model Embeddings

Ziyang Luo, Artur Kulmizev, Xiaoxi Mao

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted as long paper at ACL 2021

2003.07441 2021-05-19 cs.CV cs.LG 79%

Pretraining Image Encoders without Reconstruction via Feature Prediction Loss

Gustav Grund Pihlgren, Fredrik Sandin, Marcus Liwicki

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2105.07149 2021-05-18 cs.CL 79%

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

Qu Cui, Shujian Huang, Jiahuan Li, Xiang Geng, Zaixiang Zheng, Guoping Huang, Jiajun Chen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2105.03519 2021-05-11 cs.CL 79%

Understanding by Understanding Not: Modeling Negation in Language Models

Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, Aaron Courville

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2003.13198 2021-04-23 cs.CL cs.CV 79%

InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining

Junyang Lin, An Yang, Yichang Zhang, Jie Liu, Jingren Zhou, Hongxia Yang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages

2104.08405 2021-04-20 cs.CL cs.CV cs.IR 79%

LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding

Te-Lin Wu, Cheng Li, Mingyang Zhang, Tao Chen, Spurthi Amba Hombaiah, Michael Bendersky

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2104.07496 2021-04-16 cs.CL 79%

Unmasking the Mask -- Evaluating Social Biases in Masked Language Models

Masahiro Kaneko, Danushka Bollegala

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.04950 2021-04-13 cs.CL cs.SD eess.AS 79%

Innovative Bert-based Reranking Language Models for Speech Recognition

Shih-Hsuan Chiu, Berlin Chen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 6 pages, 3 figures, Published in IEEE SLT 2021

2102.06551 2021-04-13 cs.CL 79%

A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages

Jivnesh Sandhan, Amrith Krishna, Ashim Gupta, Laxmidhar Behera, Pawan Goyal

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 6 pages, The work is accepted at EACL-SRW, 2021, Kyiv, Ukraine Typos corrected in Section 3.2

2010.10077 2021-04-13 cs.CL 79%

Neural Language Modeling for Contextualized Temporal Graph Generation

Aman Madaan, Yiming Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments NAACL 2021

2012.06262 2021-03-29 cs.CL 79%

Morphology Matters: A Multilingual Language Modeling Analysis

Hyunji Hayley Park, Katherine J. Zhang, Coleman Haley, Kenneth Steimel, Han Liu, Lane Schwartz

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in TACL, a pre-MIT Press publication version; 15 pages, 3 figures; for the datasets, see https://github.com/hayleypark/MorphologyMatters

Journal ref Transactions of the Association for Computational Linguistics 9 (2021) 261-276

2103.08251 2021-03-16 cs.CV cs.AI 79%

Boosting ship detection in SAR images with complementary pretraining techniques

Wei Bao, Meiyu Huang, Yaqin Zhang, Yao Xu, Xuejiao Liu, Xueshuang Xiang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2103.08157 2021-03-16 cs.LG 79%

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning

Kwanghee Choi, Minyoung Choe, Hyelee Lee

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to NeurIPS 2020 Workshop: Self-Supervised Learning - Theory and Practice

2010.12688 2021-03-16 cs.CL 79%

Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training

Oshin Agarwal, Heming Ge, Siamak Shakeri, Rami Al-Rfou

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at NAACL 2021

2004.13952 2021-03-12 cs.CL 79%

Data Augmentation for Spoken Language Understanding via Pretrained Language Models

Baolin Peng, Chenguang Zhu, Michael Zeng, Jianfeng Gao

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 1 figure

2103.05131 2021-03-10 cs.CL 79%

Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data

Sanjeev Kumar Karn, Francine Chen, Yan-Ying Chen, Ulli Waltinger, Hinrich Schuetze

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Adapt-NLP: The Second Workshop on Domain Adaptation for NLP

2102.07370 2021-02-16 cs.CL 79%

Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification

Bidisha Sharma, Maulik Madhavi, Haizhou Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2102.04754 2021-02-10 cs.CL 79%

Bayesian Transformer Language Models for Speech Recognition

Boyang Xue, Jianwei Yu, Junhao Xu, Shansong Liu, Shoukang Hu, Zi Ye, Mengzhe Geng, Xunying Liu, Helen Meng

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2101.11562 2021-01-28 cs.CV cs.CL 79%

Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network

Yehao Li, Yingwei Pan, Ting Yao, Jingwen Chen, Tao Mei

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments AAAI 2021; Code is publicly available at: https://github.com/YehLi/TDEN

2101.10717 2021-01-27 cs.CL 79%

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EACL 2021

1. 预训练与数据 12393 篇

A Cognitive Regularizer for Language Modeling

Pretraining Representations for Data-Efficient Reinforcement Learning

Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

Lookup-Table Recurrent Language Models for Long Tail Speech Recognition

belabBERT: a Dutch RoBERTa-based language model applied to psychiatric classification

Implicit Representations of Meaning in Neural Language Models

Few-Shot Question Answering by Pretraining Span Selection

Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs

ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans

Positional Artefacts Propagate Through Masked Language Model Embeddings

Pretraining Image Encoders without Reconstruction via Feature Prediction Loss

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

Understanding by Understanding Not: Modeling Negation in Language Models

InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining

LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding

Unmasking the Mask -- Evaluating Social Biases in Masked Language Models

Innovative Bert-based Reranking Language Models for Speech Recognition

A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages

Neural Language Modeling for Contextualized Temporal Graph Generation

Morphology Matters: A Multilingual Language Modeling Analysis

Boosting ship detection in SAR images with complementary pretraining techniques

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning

Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training

Data Augmentation for Spoken Language Understanding via Pretrained Language Models

Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data

Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification

Bayesian Transformer Language Models for Speech Recognition

Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification