语言大模型 / LLM - arXivDaily 专题

2207.09643 2022-07-21 cs.CL 79%

Integrating Linguistic Theory and Neural Language Models

Bai Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments PhD dissertation

2204.02393 2022-07-19 cs.CV cs.LG cs.RO 79%

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

Qihang Zhang, Zhenghao Peng, Bolei Zhou

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ECCV accepted paper

2207.07611 2022-07-18 cs.LG cs.CV cs.SD eess.AS 79%

Position Prediction as an Effective Pretraining Strategy

Shuangfei Zhai, Navdeep Jaitly, Jason Ramapuram, Dan Busbridge, Tatiana Likhomanenko, Joseph Yitan Cheng, Walter Talbott, Chen Huang, Hanlin Goh, Joshua Susskind

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to ICML 2022

2207.07116 2022-07-15 cs.CV cs.LG 79%

Bootstrapped Masked Autoencoders for Vision BERT Pretraining

Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ECCV 2022, code is available at https://github.com/LightDXY/BootMAE

2207.06911 2022-07-15 eess.SP cs.LG 79%

Improving self-supervised pretraining models for epileptic seizure detection from EEG data

Sudip Das, Pankaj Pandey, Krishna Prasad Miyapuram

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2201.07219 2022-07-15 eess.IV cs.CV cs.LG 79%

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Mohamed Saeed, Rand Muhtaseb, Mohammad Yaqub

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2207.03208 2022-07-13 cs.LG 79%

Revisiting Pretraining Objectives for Tabular Deep Learning

Ivan Rubachev, Artem Alekberov, Yury Gorishniy, Artem Babenko

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Code: https://github.com/puhsu/tabular-dl-pretrain-objectives

2207.03637 2022-07-11 cs.CL 79%

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

Zhengbao Jiang, Yi Mao, Pengcheng He, Graham Neubig, Weizhu Chen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments NAACL 2022

2207.01718 2022-07-06 cs.CL eess.AS 79%

BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model

Brooke Stephenson, Laurent Besacier, Laurent Girin, Thomas Hueber

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages

2204.05188 2022-07-04 cs.CL cs.SD eess.AS 79%

Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems

Vishal Sunder, Eric Fosler-Lussier, Samuel Thomas, Hong-Kwang J. Kuo, Brian Kingsbury

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 5 pages, 2 figures

2109.14989 2022-06-30 cs.CL 79%

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

Arabella Sinclair, Jaap Jumelet, Willem Zuidema, Raquel Fernández

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Published in TACL, MIT Press

2203.09690 2022-06-22 eess.AS cs.CL cs.SD 79%

A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing

He Bai, Renjie Zheng, Junkun Chen, Xintong Li, Mingbo Ma, Liang Huang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ICML 2022, 12 pages, 10 figures

2110.14378 2022-06-09 cs.AI 79%

Towards artificial general intelligence via a multimodal foundation model

Nanyi Fei, Zhiwu Lu, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Hao Sun, Ji-Rong Wen

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

Comments Published by Nature Communications, see https://www.nature.com/articles/s41467-022-30761-2

2110.13492 2022-06-08 cs.LG cs.SD eess.AS 79%

TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining

Viet-Anh Nguyen, Anh H. T. Nguyen, Andy W. H. Khong

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Published as a conference paper at ICASSP 2022, 5 pages, 4 figures, 3 tables

2205.13607 2022-06-06 cs.LG cs.HC 79%

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

Mike A. Merrill, Tim Althoff

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2205.10233 2022-06-06 cs.CL 79%

RigoBERTa: A State-of-the-Art Language Model For Spanish

Alejandro Vaca Serrano, Guillem Garcia Subies, Helena Montoro Zamorano, Nuria Aldama Garcia, Doaa Samy, David Betancur Sanchez, Antonio Moreno Sandoval, Marta Guerrero Nieto, Alvaro Barbero Jimenez

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2012.05776 2022-06-02 cs.CL 79%

Multi-Sense Language Modelling

Andrea Lekkas, Peter Schneider-Kamp, Isabelle Augenstein

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2205.07083 2022-05-17 eess.AS cs.CL 79%

Pretraining Approaches for Spoken Language Recognition: TalTech Submission to the OLR 2021 Challenge

Tanel Alumäe, Kunnar Kukk

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted to Speaker Odyssey 2022

2205.03403 2022-05-09 cs.CL 79%

A Data Cartography based MixUp for Pre-trained Language Models

Seo Yeon Park, Cornelia Caragea

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at NAACL 2022 main conference. arXiv admin note: text overlap with arXiv:2203.07559

2110.08743 2022-05-05 cs.CL 79%

GNN-LM: Language Modeling based on Global Contexts via GNN

Yuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun, Tianwei Zhang, Fei Wu, Jiwei Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear at ICLR 2022

2202.02093 2022-05-05 cs.CL 79%

Temporal Attention for Language Models

Guy D. Rosin, Kira Radinsky

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of NAACL 2022. 9 pages

2205.00320 2022-05-03 cs.CL 79%

Detoxifying Language Models with a Toxic Corpus

Yoon A Park, Frank Rudzicz

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2204.10939 2022-04-29 cs.CL cs.CV 79%

Unified Pretraining Framework for Document Understanding

Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Nikolaos Barmpalios, Rajiv Jain, Ani Nenkova, Tong Sun

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 12 pages, 4 figures, NeurIPS 2021 (Updated Camera Ready)

2204.11989 2022-04-27 cs.IR cs.CL 79%

C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval

Eugene Yang, Suraj Nair, Ramraj Chandradevan, Rebecca Iglesias-Flores, Douglas W. Oard

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 6 pages, 2 figures, accepted as a SIGIR 2022 Short Paper

2204.09653 2022-04-21 cs.PL cs.CL cs.SE 79%

On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages

Fuxiang Chen, Fatemeh Fard, David Lo, Timofey Bryksin

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted in ICPC 2022

2204.06130 2022-04-21 cs.CL 79%

Impossible Triangle: What's Next for Pre-trained Language Models?

Chenguang Zhu, Michael Zeng

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2110.08547 2022-04-14 cs.CL 79%

Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation

Guanhua Chen, Shuming Ma, Yun Chen, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2022. Code will be available at [https://github.com/ghchen18/acl22-sixtp]

2204.05356 2022-04-13 cs.CL 79%

A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis

Ehsan Hosseini-Asl, Wenhao Liu, Caiming Xiong

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to Findings of NAACL 2022

2204.01440 2022-04-05 cs.CL cs.CY 79%

Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study

Serra Sinem Tekiroglu, Helena Bonaldi, Margherita Fanton, Marco Guerini

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL): Findings"

2109.07323 2022-03-29 cs.IR cs.LG 79%

FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining

Zhoujun Cheng, Haoyu Dong, Ran Jia, Pengfei Wu, Shi Han, Fan Cheng, Dongmei Zhang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted by ACL'22 main track

1. 预训练与数据 12393 篇

Integrating Linguistic Theory and Neural Language Models

Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining

Position Prediction as an Effective Pretraining Strategy

Bootstrapped Masked Autoencoders for Vision BERT Pretraining

Improving self-supervised pretraining models for epileptic seizure detection from EEG data

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Revisiting Pretraining Objectives for Tabular Deep Learning

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model

Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

A$^3$T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing

Towards artificial general intelligence via a multimodal foundation model

TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

RigoBERTa: A State-of-the-Art Language Model For Spanish

Multi-Sense Language Modelling

Pretraining Approaches for Spoken Language Recognition: TalTech Submission to the OLR 2021 Challenge

A Data Cartography based MixUp for Pre-trained Language Models

GNN-LM: Language Modeling based on Global Contexts via GNN

Temporal Attention for Language Models

Detoxifying Language Models with a Toxic Corpus

Unified Pretraining Framework for Document Understanding

C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval

On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages

Impossible Triangle: What's Next for Pre-trained Language Models?

Towards Making the Most of Multilingual Pretraining for Zero-Shot Neural Machine Translation

A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis

Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study

FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining