语言大模型 / LLM

Encoder-Decoder Networks for Self-Supervised Pretraining and Downstream Signal Bandwidth Regression on Digital Antenna Arrays

Rajib Bhattacharjea, Nathan West

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

A Language Model for Grammatical Error Correction in L2 Russian

Nikita Remnev, Sergei Obiedkov, Ekaterina Rakhilina, Ivan Smirnov, Anastasia Vyrenkova

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation

Cuong Q. Nguyen, Dante Pertusi, Kim M. Branson

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments ICML 2023 Workshop on Computational Biology

Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues

Chuyuan Li, Patrick Huber, Wen Xiao, Maxime Amblard, Chloé Braud, Giuseppe Carenini

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Journal ref Findings of the Association for Computational Linguistics: EACL 2023 (2023) 2562--2579

Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks

Masataka Kawai, Noriaki Ota, Shinsuke Yamaoka

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 20 pages, 6 figures

On the N-gram Approximation of Pre-trained Language Models

Aravind Krishnan, Jesujoba Alabi, Dietrich Klakow

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted at Interspeech 2023

RoBERTweet: A BERT Language Model for Romanian Tweets

Iulian-Marius Tăiatu, Andrei-Marius Avram, Dumitru-Clementin Cercel, Florin Pop

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted at NLDB2023

Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization

Artidoro Pagnoni, Alexander R. Fabbri, Wojciech Kryściński, Chien-Sheng Wu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments To appear at ACL 2023

10 Security and Privacy Problems in Large Foundation Models

Jinyuan Jia, Hongbin Liu, Neil Zhenqiang Gong

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments A book chapter

Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarization

Kohei Matsuura, Takanori Ashihara, Takafumi Moriya, Tomohiro Tanaka, Takatomo Kano, Atsunori Ogawa, Marc Delcroix

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted by Interspeech 2023

BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

Shibo Hao, Bowen Tan, Kaiwen Tang, Bin Ni, Xiyan Shao, Hengzhe Zhang, Eric P. Xing, Zhiting Hu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments ACL 2023 (Findings); Code available at https://github.com/tanyuqian/knowledge-harvest-from-lms

"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning

Abisek Rajakumar Kalarani, Pushpak Bhattacharyya, Niyati Chhaya, Sumit Shekhar

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases

Aaron Mueller, Tal Linzen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2023

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

Vijeta Deshpande, Dan Pechi, Shree Thatte, Vladislav Lialin, Anna Rumshisky

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2023 Findings

Multi-Document Summarization with Centroid-Based Pretraining

Ratish Puduppully, Parag Jain, Nancy F. Chen, Mark Steedman

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments ACL 2023 camera-ready

Learning from Children: Improving Image-Caption Pretraining via Curriculum

Hammad A. Ayyubi, Rahul Lokesh, Alireza Zareian, Bo Wu, Shih-Fu Chang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments ACL Findings 2023

Transformer Language Models Handle Word Frequency in Prediction Head

Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments 11 pages, 12 figures, accepted to ACL 2023 Findings (short paper)

Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages

Tomasz Limisiewicz, Jiří Balhar, David Mareček

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments in ACL Findings 2023

Commonsense Knowledge Graph Completion Via Contrastive Pretraining and Node Clustering

Siwei Wu, Xiangqing Shen, Rui Xia

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Findings of ACL 2023

Future-conditioned Unsupervised Pretraining for Decision Transformer

Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, Shuai Li

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 17 pages, 9 figures, ICML 2023

Training Data Extraction From Pre-trained Language Models: A Survey

Shotaro Ishihara

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments TrustNLP workshop at ACL 2023

Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining

Holger Severin Bovbjerg, Zheng-Hua Tan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments To be published at ICASSP2023 Workshop on Self-supervision in Audio, Speech and Beyond, 10th of June 2023, Rhodes, Greece. Copyright (c) 2023 IEEE. 5 pages, 3 figures, 3 tables

Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David R. Mortensen, Noah A. Smith, Yulia Tsvetkov

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

An Yang, Junshu Pan, Junyang Lin, Rui Men, Yichang Zhang, Jingren Zhou, Chang Zhou

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition

Siyuan Feng, Ming Tu, Rui Xia, Chuanzeng Huang, Yuxuan Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Accepted for publication in INTERSPEECH 2023

Self-Supervised Pretraining on Paired Sequences of fMRI Data for Transfer Learning to Brain Decoding Tasks

Sean Paulsen, Michael Casey

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Preprint - Accepted to International Conference on Pattern Recognition, Machine Learning and Consciousness 2023

Comprehensive Solution Program Centric Pretraining for Table-and-Text Hybrid Numerical Reasoning

Qianying Liu, Dongsheng Yang, Wenjie Zhong, Fei Cheng, Sadao Kurohashi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages

IUST_NLP at SemEval-2023 Task 10: Explainable Detecting Sexism with Transformers and Task-adaptive Pretraining

Hadiseh Mahmoudi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 6 pages, 7 figures, Accepted at The 17th International Workshop on Semantic Evaluation, ACL 2023

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video

Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, Guohai Xu, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.CL

Journal ref ICML2023

XTab: Cross-table Pretraining for Tabular Transformers

Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, Mahsa Shoaran

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG