语言大模型 / LLM - arXivDaily 专题

2203.10929 2022-12-08 cs.CL 79%

General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining

Qi Lv, Ziqiang Cao, Lei Geng, Chunhui Ai, Xu Yan, Guohong Fu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2211.13337 2022-12-07 cs.LG 79%

Multi-Environment Pretraining Enables Transfer to Action Limited Datasets

David Venuto, Sherry Yang, Pieter Abbeel, Doina Precup, Igor Mordatch, Ofir Nachum

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2211.03545 2022-12-06 eess.AS cs.CL cs.SD 79%

ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech

Xiaoran Fan, Chao Pang, Tian Yuan, He Bai, Renjie Zheng, Pengfei Zhu, Shuohuan Wang, Junkun Chen, Zeyu Chen, Liang Huang, Yu Sun, Hua Wu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2212.01681 2022-12-06 cs.CL cs.MA 79%

Language Models as Agent Models

Jacob Andreas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2207.08824 2022-11-30 q-bio.QM cs.LG 79%

Energy-Motivated Equivariant Pretraining for 3D Molecular Graphs

Rui Jiao, Jiaqi Han, Wenbing Huang, Yu Rong, Yang Liu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments AAAI 2023

2211.14429 2022-11-29 physics.chem-ph cs.LG q-bio.BM 79%

Supervised Pretraining for Molecular Force Fields and Properties Prediction

Xiang Gao, Weihao Gao, Wenzhi Xiao, Zhirui Wang, Chong Wang, Liang Xiang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments AI4Science Workshop at NeurIPS 2022

2112.08462 2022-11-28 cs.CL 79%

Applying SoftTriple Loss for Supervised Language Model Fine Tuning

Witold Sosnowski, Anna Wroblewska, Piotr Gawrysiak

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref 17th Conference on Computer Science and Intelligence Systems 2022. Series: ACSIS Annals of Computer Science and Information Systems

2211.10330 2022-11-21 cs.CL 79%

GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation

Biyang Guo, Yeyun Gong, Yelong Shen, Songqiao Han, Hailiang Huang, Nan Duan, Weizhu Chen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 21 pages

2211.04598 2022-11-10 cs.LG physics.atm-clus 79%

Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Accelerators

Jenna A. Bilbrey, Kristina M. Herman, Henry Sprueill, Soritis S. Xantheas, Payel Das, Manuel Lopez Roldan, Mike Kraus, Hatem Helal, Sutanay Choudhury

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Machine Learning and the Physical Sciences Workshop at the 36th conference on Neural Information Processing Systems (NeurIPS)

2211.04281 2022-11-09 cs.CL 79%

SocioProbe: What, When, and Where Language Models Learn about Sociodemographics

Anne Lauscher, Federico Bianchi, Samuel Bowman, Dirk Hovy

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication at EMNLP 2022

2207.06010 2022-11-03 cs.LG q-bio.BM 79%

Does GNN Pretraining Help Molecular Representation?

Ruoxi Sun, Hanjun Dai, Adams Wei Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2210.17236 2022-11-01 cs.PL cs.CL cs.SE 79%

When Language Model Meets Private Library

Daoguang Zan, Bei Chen, Zeqi Lin, Bei Guan, Yongji Wang, Jian-Guang Lou

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2022 Findings

2210.16029 2022-10-31 cs.CL cs.SD eess.AS 79%

Assessing Phrase Break of ESL speech with Pre-trained Language Models

Zhiyi Wang, Shaoguang Mao, Wenshan Wu, Yan Xia

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Under Review, ICASSP 2023

2210.14330 2022-10-27 q-bio.QM cs.AI q-bio.MN 79%

A single-cell gene expression language model

William Connell, Umair Khan, Michael J. Keiser

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.AI

Comments 10 pages, 5 figures, Accepted at Learning Meaningful Representations of Life Workshop, 36th Conference on Neural Information Processing Systems (NeurIPS 2022)

2205.11482 2022-10-26 cs.CL cs.IR 79%

Towards Tracing Factual Knowledge in Language Models Back to the Training Data

Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of EMNLP, 2022

2210.12694 2022-10-25 cs.CL 79%

Do Language Models Understand Measurements?

Sungjin Park, Seungwoo Ryu, Edward Choi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2022

2210.11165 2022-10-21 cs.CL 79%

Pre-training Language Models with Deterministic Factual Knowledge

Shaobo Li, Xiaoguang Li, Lifeng Shang, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP 2022

2210.06525 2022-10-14 cs.CL 79%

Subword Segmental Language Modelling for Nguni Languages

Francois Meyer, Jan Buys

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2206.01670 2022-10-14 cs.CV cs.AI 79%

Egocentric Video-Language Pretraining

Kevin Qinghong Lin, Alex Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Zhongcong Xu, Difei Gao, Rongcheng Tu, Wenzhe Zhao, Weijie Kong, Chengfei Cai, Hongfa Wang, Dima Damen, Bernard Ghanem, Wei Liu, Mike Zheng Shou

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by NeurIPS 2022. Double champions at Ego4D and EPIC-Kitchens, CVPR 2022 challenges. 23 pages, 13 figures, 12 tables. Code: https://github.com/showlab/EgoVLP

2210.05497 2022-10-12 cs.CL 79%

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models

Qihuang Zhong, Liang Ding, Li Shen, Peng Mi, Juhua Liu, Bo Du, Dacheng Tao

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2022 (Findings)

2205.12662 2022-10-11 cs.CL 79%

DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

Zhi Chen, Jijia Bao, Lu Chen, Yuncong Liu, Da Ma, Bei Chen, Mengyue Wu, Su Zhu, Xin Dong, Fujiang Ge, Qingliang Miao, Jian-Guang Lou, Kai Yu

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.CL

Comments Work in Progress

2210.02509 2022-10-07 cs.CL 79%

Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation

Arturo Oncevay, Kervy Dante Rivas Rojas, Liz Karen Chavez Sanchez, Roberto Zariquiey

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments COLING 2022, short-paper

2209.15101 2022-10-03 cs.LG physics.chem-ph q-bio.BM 79%

Improving Molecular Pretraining with Complementary Featurizations

Yanqiao Zhu, Dingshuo Chen, Yuanqi Du, Yingze Wang, Qiang Liu, Shu Wu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 24 pages, work in progress

2209.11102 2022-09-23 cs.CL 79%

Scope of Pre-trained Language Models for Detecting Conflicting Health Information

Joseph Gatto, Madhusudan Basak, Sarah M. Preum

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2209.07068 2022-09-16 cs.CL 79%

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

Piji Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments COLING2022,11 pages

2208.11663 2022-08-25 cs.CL 79%

PEER: A Collaborative Language Model

Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, Sebastian Riedel

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2208.07864 2022-08-18 cs.CL 79%

BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification

Vinura Dhananjaya, Piyumal Demotte, Surangika Ranathunga, Sanath Jayasena

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2203.05140 2022-08-15 cs.CL 79%

Speciesist Language and Nonhuman Animal Bias in English Masked Language Models

Masashi Takeshita, Rafal Rzepka, Kenji Araki

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments This paper is an accepted manuscript for publication in Information Processing & Management

2203.11709 2022-08-10 cs.CV cs.AI 79%

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

Feng Wang, Huiyu Wang, Chen Wei, Alan Yuille, Wei Shen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2208.01857 2022-08-04 cs.LG math.OC stat.ML 79%

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift

Jingfeng Wu, Difan Zou, Vladimir Braverman, Quanquan Gu, Sham M. Kakade

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 32 pages, 1 figure, 1 table

1. 预训练与数据 12393 篇

General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining

Multi-Environment Pretraining Enables Transfer to Action Limited Datasets

ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech

Language Models as Agent Models

Energy-Motivated Equivariant Pretraining for 3D Molecular Graphs

Supervised Pretraining for Molecular Force Fields and Properties Prediction

Applying SoftTriple Loss for Supervised Language Model Fine Tuning

GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation

Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Accelerators

SocioProbe: What, When, and Where Language Models Learn about Sociodemographics

Does GNN Pretraining Help Molecular Representation?

When Language Model Meets Private Library

Assessing Phrase Break of ESL speech with Pre-trained Language Models

A single-cell gene expression language model

Towards Tracing Factual Knowledge in Language Models Back to the Training Data

Do Language Models Understand Measurements?

Pre-training Language Models with Deterministic Factual Knowledge

Subword Segmental Language Modelling for Nguni Languages

Egocentric Video-Language Pretraining

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models

DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation

Improving Molecular Pretraining with Complementary Featurizations

Scope of Pre-trained Language Models for Detecting Conflicting Health Information

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

PEER: A Collaborative Language Model

BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification

Speciesist Language and Nonhuman Animal Bias in English Masked Language Models

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift