语言大模型 / LLM

Testing the limits of natural language models for predicting human language judgments

Tal Golan, Matthew Siegelman, Nikolaus Kriegeskorte, Christopher Baldassano

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Journal ref Nature Machine Intelligence (2023)

Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy

Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers

Dahun Kim, Anelia Angelova, Weicheng Kuo

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments CVPR 2023 Highlight - https://github.com/mcahny/rovit ; adds LAION-2B result

Generative Pretraining for Black-Box Optimization

Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments International Conference for Machine Learning 2023 NeurIPS Workshop for Foundational Models for Decision Making (Oral) 2022

OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

MAP: A Model-agnostic Pretraining Framework for Click-through Rate Prediction

Jianghao Lin, Yanru Qu, Wei Guo, Xinyi Dai, Ruiming Tang, Yong Yu, Weinan Zhang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments KDD 2023

Opinion Mining Using Population-tuned Generative Language Models

Allmin Susaiyah, Abhinay Pandya, Aki Härmä

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Making Pre-trained Language Models both Task-solvers and Self-calibrators

Yangyi Chen, Xingyao Wang, Heng Ji

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to Findings of ACL 2023

Enhancing LLM with Evolutionary Fine Tuning for News Summary Generation

Le Xiao, Xiaolin Chen

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 2 figures

Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentials

Yuyang Wang, Changwen Xu, Zijie Li, Amir Barati Farimani

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Published in Journal of Chemical Theory and Computation. 32 pages, 5 figures, 2 tables

Hierarchical Pretraining for Biomedical Term Embeddings

Bryan Cai, Sihang Zeng, Yucong Lin, Zheng Yuan, Doudou Zhou, Lu Tian

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.CL、cs.AI

Investigating Masking-based Data Generation in Language Models

Ed S. Ma

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Work in progress

Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting

Zahra Fatemi, Chen Xing, Wenhao Liu, Caiming Xiong

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments This paper has been accepted at the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)

Supervised Pretraining Can Learn In-Context Reinforcement Learning

Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis

Juhua Liu, Qihuang Zhong, Liang Ding, Hua Jin, Bo Du, Dacheng Tao

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by IEEE TASLP 2023

DeepJoin: Joinable Table Discovery with Pre-trained Language Models

Yuyang Dong, Chuan Xiao, Takuma Nozawa, Masafumi Enomoto, Masafumi Oyamada

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Journal ref VLDB2023

VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations

Tiancheng Zhao, Tianqi Zhang, Mingwei Zhu, Haozhan Shen, Kyusong Lee, Xiaopeng Lu, Jianwei Yin

专题命中 预训练与数据 ：language model(title)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 9 pages, preprint

MolCAP: Molecular Chemical reActivity pretraining and prompted-finetuning enhanced molecular representation learning

Yu Wang, JingJie Zhang, Junru Jin, Leyi Wei

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Privately generating tabular data using language models

Alexandre Sablayrolles, Yue Wang, Brian Karrer

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 3 figures

The Effectiveness of Bidirectional Generative Patent Language Models

Jieh-Sheng Lee

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 10 pages, 3 figures, 5 tables

Journal ref Volume 362: Legal Knowledge and Information Systems, 2022

How Does Pretraining Improve Discourse-Aware Translation?

Zhihong Huang, Longyue Wang, Siyou Liu, Derek F. Wong

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.CL、cs.AI

Comments Interspeech 2023

What does the Failure to Reason with "Respectively" in Zero/Few-Shot Settings Tell Us about Language Models?

Ruixiang Cui, Seolhwa Lee, Daniel Hershcovich, Anders Søgaard

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments To appear at ACL 2023

Likelihood-Based Diffusion Language Models

Ishaan Gulrajani, Tatsunori B. Hashimoto

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Downstream Datasets Make Surprisingly Good Pretraining Corpora

Kundan Krishna, Saurabh Garg, Jeffrey P. Bigham, Zachary C. Lipton

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments ACL2023 Camera Ready

Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models

Pengcheng Jiang, Shivam Agarwal, Bowen Jin, Xuan Wang, Jimeng Sun, Jiawei Han

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 11 figures, 8 tables. Accepted by ACL 23' Findings

Using Language Models to Detect Alarming Student Responses

Christopher M. Ormerod, Milan Patel, Harry Wang

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 3 figures

Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning

Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, Tarek Abdelzaher, Jiawei Han

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ICML 2023. (Code: https://github.com/yumeng5/FewGen)

Pretraining Without Attention

Junxiong Wang, Jing Nathan Yan, Albert Gu, Alexander M. Rush

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

A Large and Diverse Arabic Corpus for Language Modeling

Abbas Raza Ali, Muhammad Ajmal Siddiqui, Rema Algunaibet, Hasan Raza Ali

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness

Yichuan Li, Jialong Han, Kyumin Lee, Chengyuan Ma, Benjamin Yao, Derek Liu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI