语言大模型 / LLM - arXivDaily 专题

2302.00539 2023-04-25 cs.LG 79%

Analyzing Leakage of Personally Identifiable Information in Language Models

Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, Santiago Zanella-Béguelin

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments IEEE Symposium on Security and Privacy (S&P) 2023

2304.10311 2023-04-21 cs.MM cs.LG 79%

Movie Box Office Prediction With Self-Supervised and Visually Grounded Pretraining

Qin Chao, Eunsoo Kim, Boyang Li

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments accepted by IEEE International Conference on Multimedia and Expo (2023)

2304.07854 2023-04-18 cs.CL 79%

Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation

Yunjie Ji, Yan Gong, Yong Deng, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2304.05369 2023-04-12 cs.LG 79%

A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation

Florian Bordes, Samuel Lavoie, Randall Balestriero, Nicolas Ballas, Pascal Vincent

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2304.02724 2023-04-07 cs.CV cs.LG 79%

Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound

Blake VanBerlo, Brian Li, Alexander Wong, Jesse Hoey, Robert Arntfield

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 10 pages, 6 figures, to be published in the 2023 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

2212.04231 2023-03-30 cs.CV cs.CL 79%

Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations

Björn Plüster, Jakob Ambsdorf, Lukas Braach, Jae Hee Lee, Stefan Wermter

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Minor changes

2303.14382 2023-03-28 cs.CV cs.LG 79%

Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm

Yichen Xie, Han Lu, Junchi Yan, Xiaokang Yang, Masayoshi Tomizuka, Wei Zhan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted by CVPR 2023

2207.14251 2023-03-27 cs.CL 79%

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments We received a criticism regarding the validity of the causal formulation in this paper. We will address them in an upcoming version

2303.13076 2023-03-24 cs.CV cs.AI 79%

CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching

Xiaoshi Wu, Feng Zhu, Rui Zhao, Hongsheng Li

专题命中预训练与数据：prompting(title,abstract)；分类 cs.AI

Comments 11 pages, 4 figures. Accepted by CVPR 2023

2211.03053 2023-03-15 cs.CL 79%

Suffix Retrieval-Augmented Language Modeling

Zecheng Wang, Yik-Cheung Tam

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 1 figure. Submitted to ICASSP 2023

2303.03457 2023-03-08 cs.CL 79%

Spelling convention sensitivity in neural language models

Elizabeth Nielsen, Christo Kirov, Brian Roark

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref EACL Findings 2023

2303.03144 2023-03-07 cs.MM cs.CL cs.SD eess.AS 79%

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages, 8 figures, 5 Tables

2210.07468 2023-03-07 cs.CL 79%

Transparency Helps Reveal When Language Models Learn Meaning

Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2023. Author's final version (pre-MIT Press publication)

2303.01566 2023-03-06 stat.ML cs.LG math.ST stat.TH 79%

On the Provable Advantage of Unsupervised Pretraining

Jiawei Ge, Shange Tang, Jianqing Fan, Chi Jin

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2206.13602 2023-03-02 cs.LG q-bio.QM 79%

Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

Shengchao Liu, Hongyu Guo, Jian Tang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.14389 2023-03-01 cs.CL 79%

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion, Christophe Pallier

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 19 pages, 8 figures, 10 pages of Appendix, 5 appendix figures

2302.14141 2023-03-01 cs.LG 79%

Linear pretraining in recurrent mixture density networks

Hubert Normandin-Taillon, Frédéric Godin, Chun Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.09483 2023-02-21 cs.LG 79%

Why Is Public Pretraining Necessary for Private Model Training?

Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Thakurta, Lun Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.08427 2023-02-17 cs.CV cs.AI 79%

Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies

Emma Sarfati, Alexandre Bone, Marc-Michel Rohe, Pietro Gori, Isabelle Bloch

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted at IEEE ISBI 2023

2210.07587 2023-02-14 cs.CL 79%

ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining

Ranran Haoran Zhang, Aysa Xuemo Fan, Rui Zhang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by EACL 2023

2206.03826 2023-02-14 cs.LG cs.CV cs.NE stat.ML 79%

Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Tasks

Jiachun Pan, Pan Zhou, Shuicheng Yan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2208.03219 2023-02-07 cs.CL 79%

Construction of English Resume Corpus and Test with Pre-trained Language Models

Chengguang Gan, Tatsunori Mori

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments The Association for Natural Language Processing in Japan. Submit in NLP2023

2302.01483 2023-02-06 cs.LG cs.SD eess.AS 79%

SPADE: Self-supervised Pretraining for Acoustic DisEntanglement

John Harvill, Jarred Barber, Arun Nair, Ramin Pishehvar

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2207.04993 2023-02-01 cs.CL 79%

Embedding Recycling for Language Models

Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug Downey

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EACL Findings 2023

2301.02111 2023-01-06 cs.CL cs.SD eess.AS 79%

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers

Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Working in progress

2301.00447 2023-01-03 cs.CV cs.LG 79%

Image To Tree with Recursive Prompting

James Batten, Matthew Sinclair, Ben Glocker, Michiel Schaap

专题命中预训练与数据：prompting(title,abstract)；分类 cs.LG

Comments 12 pages, 5 figures

2207.13988 2023-01-03 cs.CL 79%

Sequence to sequence pretraining for a less-resourced Slovenian language

Matej Ulčar, Marko Robnik-Šikonja

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 19 pages

2212.08967 2022-12-20 cs.AI 79%

Foundation models in brief: A historical, socio-technical focus

Johannes Schneider

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2212.08619 2022-12-19 cs.CL cs.CR 79%

Planting and Mitigating Memorized Content in Predictive-Text Language Models

C. M. Downey, Wei Dai, Huseyin A. Inan, Kim Laine, Saurabh Naik, Tomasz Religa

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2212.08321 2022-12-19 eess.AS cs.CL 79%

Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language

Yusuke Yasuda, Tomoki Toda

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.CL

Journal ref IEEE Journal of Selected Topics in Signal Processing (Volume: 16, Issue: 6, October 2022)

1. 预训练与数据 12393 篇

Analyzing Leakage of Personally Identifiable Information in Language Models

Movie Box Office Prediction With Self-Supervised and Visually Grounded Pretraining

Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation

A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation

Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound

Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations

Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching

Suffix Retrieval-Augmented Language Modeling

Spelling convention sensitivity in neural language models

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining

Transparency Helps Reveal When Language Models Learn Meaning

On the Provable Advantage of Unsupervised Pretraining

Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

Linear pretraining in recurrent mixture density networks

Why Is Public Pretraining Necessary for Private Model Training?

Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies

ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining

Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Tasks

Construction of English Resume Corpus and Test with Pre-trained Language Models

SPADE: Self-supervised Pretraining for Acoustic DisEntanglement

Embedding Recycling for Language Models

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers

Image To Tree with Recursive Prompting

Sequence to sequence pretraining for a less-resourced Slovenian language

Foundation models in brief: A historical, socio-technical focus

Planting and Mitigating Memorized Content in Predictive-Text Language Models

Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language