语言大模型 / LLM - arXivDaily 专题

2109.15300 2021-10-01 cs.CL 79%

Semi-Supervised Text Classification via Self-Pretraining

Payam Karisani, Negin Karisani

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments WSDM 2021

2109.12383 2021-09-28 cs.CL 79%

Language Model Priming for Cross-Lingual Event Extraction

Steven Fincke, Shantanu Agarwal, Scott Miller, Elizabeth Boschee

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2109.12028 2021-09-27 cs.CL 79%

Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering

Fahim Faisal, Antonios Anastasopoulos

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments Accepted at MRQA Workshop 2021

2109.11778 2021-09-27 cs.CV cs.CL 79%

Dense Contrastive Visual-Linguistic Pretraining

Lei Shi, Kai Shuang, Shijie Geng, Peng Gao, Zuohui Fu, Gerard de Melo, Yunpeng Chen, Sen Su

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ACM Multimedia 2021. arXiv admin note: text overlap with arXiv:2007.13135

2109.08354 2021-09-20 cs.CL 79%

Task-adaptive Pre-training of Language Models with Word Embedding Regularization

Kosuke Nishida, Kyosuke Nishida, Sen Yoshida

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL Findings 2021

2109.08249 2021-09-20 cs.CL 79%

Regularized Training of Nearest Neighbor Language Models

Jean-Francois Ton, Walter Talbott, Shuangfei Zhai, Josh Susskind

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2109.07765 2021-09-17 cs.CL 79%

Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Models

Casimiro Pio Carrino, Jordi Armengol-Estapé, Ona de Gibert Bonet, Asier Gutiérrez-Fandiño, Aitor Gonzalez-Agirre, Martin Krallinger, Marta Villegas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2108.06209 2021-09-15 cs.LG cs.SD eess.AS 79%

W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training

Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, Yonghui Wu

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2102.05766 2021-09-15 cs.CL 79%

Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation

Renjie Zheng, Junkun Chen, Mingbo Ma, Liang Huang

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

2109.04080 2021-09-14 cs.CL 79%

Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining

Yicheng Zou, Bolin Zhu, Xingwu Hu, Tao Gui, Qi Zhang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2021, 12 pages

2109.04715 2021-09-13 cs.CL 79%

AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages

Machel Reid, Junjie Hu, Graham Neubig, Yutaka Matsuo

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP 2021

2109.04666 2021-09-13 cs.CL 79%

Euphemistic Phrase Detection by Masked Language Model

Wanzheng Zhu, Suma Bhat

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.07829 2021-09-07 cs.CL 79%

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

C. M. Downey, Fei Xia, Gina-Anne Levow, Shane Steinert-Threlkeld

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2103.06678 2021-09-07 cs.CL 79%

The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models

Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, Nizar Habash

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to WANLP 2021

2011.13066 2021-09-03 cs.CV cs.AI 79%

USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning

Yixiong Chen, Chunhui Zhang, Li Liu, Cheng Feng, Changfeng Dong, Yongfang Luo, Xiang Wan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2108.13956 2021-09-01 cs.LG 79%

APS: Active Pretraining with Successor Features

Hao Liu, Pieter Abbeel

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Appeared in ICML 2021

2108.12216 2021-08-30 cs.CL 79%

Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors

Ryo Nagata, Manabu Kimura, Kazuaki Hanawa

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2108.10986 2021-08-26 cs.CL 79%

Using BERT Encoding and Sentence-Level Language Model for Sentence Ordering

Melika Golestani, Seyedeh Zahra Razavi, Zeinab Borhanifard, Farnaz Tahmasebian, Hesham Faili

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 12 pages, 2 figures, The 24th International Conference of Text, Speech and Dialogue (TSD2021)

2108.09814 2021-08-24 cs.CL 79%

UzBERT: pretraining a BERT model for Uzbek

B. Mansurov, A. Mansurov

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 9 pages, 1 table

2108.05540 2021-08-13 cs.IR cs.CL 79%

Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval

Luyu Gao, Jamie Callan

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.10972 2021-08-06 cs.CV cs.LG 79%

ImageNet-21K Pretraining for the Masses

Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, Lihi Zelnik-Manor

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to NeurIPS 2021 (Datasets and Benchmarks)

2107.12369 2021-07-27 cs.CV cs.LG 79%

Improve Unsupervised Pretraining for Few-label Transfer

Suichan Li, Dongdong Chen, Yinpeng Chen, Lu Yuan, Lei Zhang, Qi Chu, Bin Liu, Nenghai Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ICCV 2021. arXiv admin note: substantial text overlap with arXiv:2012.05899

2106.02965 2021-07-26 cs.LG cs.FL 79%

Extracting Weighted Automata for Approximate Minimization in Language Modelling

Clara Lacroce, Prakash Panangaden, Guillaume Rabusseau

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments Full version of ICGI 2020/21 paper, authors are listed in alphabetical order

2107.09433 2021-07-22 cs.CL 79%

Seed Words Based Data Selection for Language Model Adaptation

Roberto Gretter, Marco Matassoni, Daniele Falavigna

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 11 pages

Journal ref Proceedings of MT Summit 2021 - August 16-20, 2021

2107.08582 2021-07-20 cs.CL 79%

Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision

Ning Bian, Xianpei Han, Bo Chen, Hongyu Lin, Ben He, Le Sun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2104.08671 2021-07-07 cs.CL 79%

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset

Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, Daniel E. Ho

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments ICAIL 2021. Code & data available at https://github.com/reglab/casehold

2106.13627 2021-06-28 cs.CL 79%

Language Models are Good Translators

Shuo Wang, Zhaopeng Tu, Zhixing Tan, Wenxuan Wang, Maosong Sun, Yang Liu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 12 pages. Work in progress. An earlier verison of this manuscript is under review

2106.10124 2021-06-21 cs.LG 79%

Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining

Oriel Frigo, Rémy Brossard, David Dehaene

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 13 pages, 4 figures

2106.09065 2021-06-18 cs.CV cs.LG 79%

SPeCiaL: Self-Supervised Pretraining for Continual Learning

Lucas Caccia, Joelle Pineau

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2012.15525 2021-06-17 cs.CL 79%

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining

Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Weizhu Chen, Dayiheng Liu, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ICML 2021

1. 预训练与数据 12393 篇

Semi-Supervised Text Classification via Self-Pretraining

Language Model Priming for Cross-Lingual Event Extraction

Investigating Post-pretraining Representation Alignment for Cross-Lingual Question Answering

Dense Contrastive Visual-Linguistic Pretraining

Task-adaptive Pre-training of Language Models with Word Embedding Regularization

Regularized Training of Nearest Neighbor Language Models

Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Models

W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training

Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation

Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining

AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages

Euphemistic Phrase Detection by Masked Language Model

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models

USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning

APS: Active Pretraining with Successor Features

Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors

Using BERT Encoding and Sentence-Level Language Model for Sentence Ordering

UzBERT: pretraining a BERT model for Uzbek

Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval

ImageNet-21K Pretraining for the Masses

Improve Unsupervised Pretraining for Few-label Transfer

Extracting Weighted Automata for Approximate Minimization in Language Modelling

Seed Words Based Data Selection for Language Model Adaptation

Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset

Language Models are Good Translators

Graph Context Encoder: Graph Feature Inpainting for Graph Generation and Self-supervised Pretraining

SPeCiaL: Self-Supervised Pretraining for Continual Learning

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining