语言大模型 / LLM

HeRo: RoBERTa and Longformer Hebrew Language Models

Vitaly Shalumov, Harel Haskey

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Behavioral Cloning via Search in Video PreTraining Latent Space

Federico Malato, Florian Leopold, Amogh Raut, Ville Hautamäki, Andrew Melnik

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning

James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, Zsolt Kira

专题命中 预训练与数据 ：prompting(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023)

PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav

Ram Ramrakhya, Dhruv Batra, Erik Wijmans, Abhishek Das

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments 8 pages + supplement

Guiding Online Reinforcement Learning with Action-Free Offline Pretraining

Deyao Zhu, Yuhui Wang, Jürgen Schmidhuber, Mohamed Elhoseiny

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

AdPE: Adversarial Positional Embeddings for Pretraining Vision Transformers via MAE+

Xiao Wang, Ying Wang, Ziwei Xuan, Guo-Jun Qi

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments 9 pages, 5 figures

Duration-aware pause insertion using pre-trained language model for multi-speaker text-to-speech

Dong Yang, Tomoki Koriyama, Yuki Saito, Takaaki Saeki, Detai Xin, Hiroshi Saruwatari

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by ICASSP2023

Data-Efficient Protein 3D Geometric Pretraining via Refinement of Diffused Protein Structure Decoy

Yufei Huang, Lirong Wu, Haitao Lin, Jiangbin Zheng, Ge Wang, Stan Z. Li

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise

Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, Weizhu Chen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Previous version title -> GENIE: Large Scale Pre-training for Text Generation with Diffusion Model

Do Language Models Plagiarize?

Jooyoung Lee, Thai Le, Jinghui Chen, Dongwon Lee

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to WWW'23

ViDeBERTa: A powerful pre-trained language model for Vietnamese

Cong Dao Tran, Nhut Huy Pham, Anh Nguyen, Truong Son Hy, Tu Vu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Training-free Lexical Backdoor Attacks on Language Models

Yujin Huang, Terry Yue Zhuo, Qiongkai Xu, Han Hu, Xingliang Yuan, Chunyang Chen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to International World Wide Web Conference 2023, Security, Privacy & Trust Track

Can Foundation Models Help Us Achieve Perfect Secrecy?

Simran Arora, Christopher Ré

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Keywords: privacy-preserving, personalized machine learning, in-context learning

Context-Aware Differential Privacy for Language Modeling

My H. Dinh, Ferdinando Fioretto

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

SMART: Self-supervised Multi-task pretrAining with contRol Transformers

Yanchao Sun, Shuang Ma, Ratnesh Madaan, Rogerio Bonatti, Furong Huang, Ashish Kapoor

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretraining

Karol Nowakowski, Michal Ptaszynski, Kyoko Murasaki, Jagna Nieuważny

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments 14 pages

Journal ref Information Processing & Management, Volume 60, Issue 2, March 2023, 103148, ISSN 0306-4573

Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries

Ali Al-Kaswan, Toufique Ahmed, Maliheh Izadi, Anand Ashok Sawant, Premkumar Devanbu, Arie van Deursen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments SANER 2023 Technical Track Camera Ready

Targeted Phishing Campaigns using Large Scale Language Models

Rabimba Karanjai

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

PatternRank: Leveraging Pretrained Language Models and Part of Speech for Unsupervised Keyphrase Extraction

Tim Schopf, Simon Klimek, Florian Matthes

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to 14th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - KDIR

Journal ref In Proceedings of the 14th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - KDIR 2022, ISBN 978-989-758-614-9; ISSN 2184-3228, pages 243-248. DOI: 10.5220/0011546600003335

Language model acceptability judgements are not always robust to context

Koustuv Sinha, Jon Gauthier, Aaron Mueller, Kanishka Misra, Keren Fuentes, Roger Levy, Adina Williams

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

What Has Been Enhanced in my Knowledge-Enhanced Language Model?

Yifan Hou, Guoji Fu, Mrinmaya Sachan

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Our code, demo, and instructions of the usage can be found in https://github.com/yifan-h/GCS_KI

RobBERT-2022: Updating a Dutch Language Model to Account for Evolving Language Use

Pieter Delobelle, Thomas Winters, Bettina Berendt

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 1 figure, 3 tables

Adaptive Multi-Corpora Language Model Training for Speech Recognition

Yingyi Ma, Zhe Liu, Xuedong Zhang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Pretraining in Deep Reinforcement Learning: A Survey

Zhihui Xie, Zichuan Lin, Junyou Li, Shuai Li, Deheng Ye

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment

Tuan Dinh, Jy-yong Sohn, Shashank Rajput, Timothy Ossowski, Yifei Ming, Junjie Hu, Dimitris Papailiopoulos, Kangwook Lee

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP Findings)

PoeLM: A Meter- and Rhyme-Controllable Language Model for Unsupervised Poetry Generation

Aitor Ormazabal, Mikel Artetxe, Manex Agirrezabal, Aitor Soroa, Eneko Agirre

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP Findings 2022

Differentially Private Language Models for Secure Data Sharing

Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, Mrinmaya Sachan

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at EMNLP 2022

Exploring Mode Connectivity for Pre-trained Language Models

Yujia Qin, Cheng Qian, Jing Yi, Weize Chen, Yankai Lin, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2022, main conference

Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future

Guo-Jun Qi, Mubarak Shah

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

IDIAPers @ Causal News Corpus 2022: Extracting Cause-Effect-Signal Triplets via Pre-trained Autoregressive Language Model

Martin Fajcik, Muskaan Singh, Juan Zuluaga-Gomez, Esaú Villatoro-Tello, Sergio Burdisso, Petr Motlicek, Pavel Smrz

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Camera-ready for CASE@EMNLP