语言大模型 / LLM

Bangla-Wave: Improving Bangla Automatic Speech Recognition Utilizing N-gram Language Models

Mohammed Rakib, Md. Ismail Hossain, Nabeel Mohammed, Fuad Rahman

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

VL-Taboo: An Analysis of Attribute-based Zero-shot Capabilities of Vision-Language Models

Felix Vogel, Nina Shvetsova, Leonid Karlinsky, Hilde Kuehne

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Structure-Grounded Pretraining for Text-to-SQL

Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, Matthew Richardson

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2021. The Spider-Realistic dataset is available at https://doi.org/10.5281/zenodo.5205322

Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models

Vilém Zouhar, Marius Mosbach, Dietrich Klakow

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Submitted to PBML. Code & experiment repository: https://github.com/zouharvi/sentence-embd-fusion

Knowing Where and What: Unified Word Block Pretraining for Document Understanding

Song Tao, Zijian Wang, Tiantian Fan, Canjie Luo, Can Huang

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.CL、cs.AI

Comments incomplete experiments

LAD: Language Models as Data for Zero-Shot Dialog

Shikib Mehri, Yasemin Altun, Maxine Eskenazi

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as a long paper to SIGDial 2022

Zero-Shot Temporal Action Detection via Vision-Language Prompting

Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, Tao Xiang

专题命中 预训练与数据 ：prompting(title,abstract)；分类 cs.CL、cs.AI

Comments ECCV 2022; Code available at https://github.com/sauradip/STALE

Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps

Alexandre Pasquiou, Yair Lakretz, John Hale, Bertrand Thirion, Christophe Pallier

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Journal ref ICML 2022 - 39th International Conference on Machine Learning, Jul 2022, Baltimore, United States. pp.18

Improving Low-Resource Speech Recognition with Pretrained Speech Models: Continued Pretraining vs. Semi-Supervised Training

Mitchell DeHaven, Jayadev Billa

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments Submitted to Interspeech 2022

DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models

Gregor Betz, Kyle Richardson

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments A Demo is available at https://huggingface.co/spaces/debatelab/deepa2-demo , the model can be downloaded from https://huggingface.co/debatelab/argument-analyst , and the datasets can be accessed at https://huggingface.co/datasets/debatelab/aaac

Journal ref *SEM 2022

CoSP: Co-supervised pretraining of pocket and ligand

Zhangyang Gao, Cheng Tan, Lirong Wu, Stan Z. Li

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, Jeff Clune

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognition

W. Ronny Huang, Cal Peyser, Tara N. Sainath, Ruoming Pang, Trevor Strohman, Shankar Kumar

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Interspeech 2022

Private Language Model Adaptation for Speech Recognition

Zhe Liu, Ke Li, Shreyan Bakshi, Fuchun Peng

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval

Uri Alon, Frank F. Xu, Junxian He, Sudipta Sengupta, Dan Roth, Graham Neubig

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to ICML'2022. Code and models are available at https://github.com/neulab/retomaton

Learning to segment with limited annotations: Self-supervised pretraining with regression and contrastive loss in MRI

Lavanya Umapathy, Zhiyang Fu, Rohit Philip, Diego Martin, Maria Altbach, Ali Bilgin

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Presented at the Annual Conference of International Society for Magnetic Resonance in Medicine, London, UK. May 2022

TiBERT: Tibetan Pre-trained Language Model

Yuan Sun, Sisi Liu, Junjie Deng, Xiaobing Zhao

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

End-to-end Generative Pretraining for Multimodal Video Captioning

Paul Hongsuck Seo, Arsha Nagrani, Anurag Arnab, Cordelia Schmid

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR) 2022

UBERT: A Novel Language Model for Synonymy Prediction at Scale in the UMLS Metathesaurus

Thilini Wijesiriwardene, Vinh Nguyen, Goonmeet Bajaj, Hong Yung Yip, Vishesh Javangula, Yuqing Mao, Kin Wah Fung, Srinivasan Parthasarathy, Amit P. Sheth, Olivier Bodenreider

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Super-Prompting: Utilizing Model-Independent Contextual Data to Reduce Data Annotation Required in Visual Commonsense Tasks

Navid Rezaei, Marek Z. Reformat

专题命中 预训练与数据 ：prompting(title)；language model(abstract)；分类 cs.CL、cs.AI

Your fairness may vary: Pretrained language model fairness in toxic text classification

Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Mikhail Yurochkin, Moninder Singh

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Findings of ACL 2022

A Review on Language Models as Knowledge Bases

Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, Marjan Ghazvininejad

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Preprint

Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling

Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Daxin Jiang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages

Fusing finetuned models for better pretraining

Leshem Choshen, Elad Venezian, Noam Slonim, Yoav Katz

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design

Kourosh Hakhamaneshi, Marcel Nassar, Mariano Phielipp, Pieter Abbeel, Vladimir Stojanović

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Deduplicating Training Data Makes Language Models Better

Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to ACL 2022

Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation

Wenxuan Wang, Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu, Michael Lyu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2022 main conference

Sharpness-Aware Minimization Improves Language Model Generalization

Dara Bahri, Hossein Mobahi, Yi Tay

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ACL 2022 Main Conference

Memorisation versus Generalisation in Pre-trained Language Models

Michael Tänzer, Sebastian Ruder, Marek Rei

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 15 pages, 25 figures. To be published in ACL2022

L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources

Raviraj Joshi

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG