语言大模型 / LLM - arXivDaily 专题

2405.14225 2024-05-24 q-bio.QM cs.CL cs.MM 79%

ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining

Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, Tat-Seng Chua

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments ACL 2024 Findings, 9 pages

2405.10254 2024-05-24 eess.IV cs.CV cs.LG 79%

PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology

George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D. Kunz, Juan A. Retamero, Gerard Oakley, David Klimstra, Christopher Kanan, Matthew Hanna, Michal Zelechowski, Julian Viret, Neil Tenenholtz, James Hall, Nicolo Fusi, Razik Yousfi, Peter Hamilton, William A. Moye, Eugene Vorontsov, Siqi Liu, Thomas J. Fuchs

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2401.06620 2024-05-24 cs.CL 79%

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models

Yihong Liu, Chunlan Ma, Haotian Ye, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2024

2404.10877 2024-05-21 cs.CL 79%

Incubating Text Classifiers Following User Instruction with Nothing but LLM

Letian Peng, Jingbo Shang

专题命中预训练与数据：LLM(title,abstract)；分类 cs.CL

2405.08911 2024-05-16 cs.CV cs.LG 79%

CLIP with Quality Captions: A Strong Pretraining for Vision Tasks

Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Oncel Tuzel

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2405.08402 2024-05-15 cs.CL 79%

Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining

Valentin Vielzeuf

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2309.05494 2024-05-15 cs.CL 79%

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts

Rabindra Lamsal, Maria Rodriguez Read, Shanika Karunasekera

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref Knowledge-Based Systems, 111916 (2024)

2404.17667 2024-04-30 eess.SP cs.LG 79%

SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

Cheng Ding, Zhicheng Guo, Zhaoliang Chen, Randall J Lee, Cynthia Rudin, Xiao Hu

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2404.10112 2024-04-19 cs.CL cs.SD eess.AS 79%

PRODIS -- a speech database and a phoneme-based language model for the study of predictability effects in Polish

Zofia Malisz, Jan Foremski, Małgorzata Kul

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in the proceedings of LREC2024: Language Resources and Evaluation Conference 2024, Turin, Italy

2311.09807 2024-04-17 cs.CL 79%

The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, Chloé Clavel

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024 Findings

2311.09615 2024-04-03 cs.CL 79%

On Retrieval Augmentation and the Limitations of Language Model Training

Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson, Ollie Liu, Isabelle Lee, Dani Yogatama

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024

2404.01156 2024-04-02 cs.CV cs.AI 79%

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

Chull Hwan Song, Taebaek Hwang, Jooyoung Yoon, Shunghyun Choi, Yeong Hyeon Gu

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.AI

Comments CVPR2024 Accepted

2404.00509 2024-04-02 cs.LG cs.CV 79%

DailyMAE: Towards Pretraining Masked Autoencoders in One Day

Jiantao Wu, Shentong Mo, Sara Atito, Zhenhua Feng, Josef Kittler, Muhammad Awais

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2403.08994 2024-04-02 cs.CL 79%

Ethos: Rectifying Language Models in Orthogonal Parameter Space

Lei Gao, Yue Niu, Tingting Tang, Salman Avestimehr, Murali Annavaram

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2403.20078 2024-04-01 cs.CV cs.LG 79%

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, Bo Han

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments ICLR 2024 Spotlight

2403.18560 2024-03-28 eess.AS cs.LG cs.SD 79%

Noise-Robust Keyword Spotting through Self-supervised Pretraining

Jacob Mørk, Holger Severin Bovbjerg, Gergely Kiss, Zheng-Hua Tan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2403.16777 2024-03-26 cs.CL 79%

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?

Shaoxiong Ji, Timothee Mickus, Vincent Segonne, Jörg Tiedemann

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments LREC-COLING 2024

2310.08391 2024-03-18 stat.ML cs.LG 79%

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

Jingfeng Wu, Difan Zou, Zixiang Chen, Vladimir Braverman, Quanquan Gu, Peter L. Bartlett

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ICLR 2024 Camera Ready

2403.08693 2024-03-14 cs.CL 79%

Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages

Rik van Noord, Taja Kuzman, Peter Rupnik, Nikola Ljubešić, Miquel Esplà-Gomis, Gema Ramírez-Sánchez, Antonio Toral

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to LREC-COLING 2024 (long)

2305.12567 2024-03-11 cs.CL 79%

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Published as a conference paper at ACL 2023. 9 pages

2310.07229 2024-03-08 cs.LG 79%

ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

Bowen Gao, Yinjun Jia, Yuanle Mo, Yuyan Ni, Weiying Ma, Zhiming Ma, Yanyan Lan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2303.06965 2024-03-08 cs.LG q-bio.BM 79%

Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model

Bo Qiang, Yiran Zhou, Yuheng Ding, Ningfeng Liu, Song Song, Liangren Zhang, Bo Huang, Zhenming Liu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2402.03302 2024-03-07 eess.IV cs.CV cs.LG 79%

Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining

Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng, Shanshan Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Code and models of Swin-UMamba are publicly available at: https://github.com/JiarunLiu/Swin-UMamba

2212.10474 2024-03-05 cs.CL 79%

ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models

Jonas Belouadi, Steffen Eger

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2023 (main track)

2311.14534 2024-02-29 cs.LG 79%

Finding Foundation Models for Time Series Classification with a PreText Task

Ali Ismail-Fawaz, Maxime Devanne, Stefano Berretti, Jonathan Weber, Germain Forestier

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2402.15404 2024-02-26 cs.LG 79%

United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once

Maurice Kraus, Felix Divo, David Steinmann, Devendra Singh Dhami, Kristian Kersting

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2402.12500 2024-02-21 cs.CV cs.LG eess.IV 79%

Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Sebastian Doerrich, Tobias Archut, Francesco Di Salvo, Christian Ledig

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Accepted at 21st IEEE International Symposium on Biomedical Imaging (IEEE ISBI 2024)

2402.10430 2024-02-19 cs.CL 79%

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

Dheeraj Mekala, Alex Nguyen, Jingbo Shang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2203.08774 2024-02-09 cs.CL 79%

CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals

Scott Novotney, Sreeparna Mukherjee, Zeeshan Ahmed, Andreas Stolcke

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in Findings of ACL 2022

Journal ref Findings of ACL 2022, pp. 3368-3379

2307.04308 2024-02-02 cs.LG 79%

Towards Cross-Table Masked Pretraining for Web Data Mining

Chao Ye, Guoshan Lu, Haobo Wang, Liyao Li, Sai Wu, Gang Chen, Junbo Zhao

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to WWW 2024

1. 预训练与数据 12393 篇

ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining

PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models

Incubating Text Classifiers Following User Instruction with Nothing but LLM

CLIP with Quality Captions: A Strong Pretraining for Vision Tasks

Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts

SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

PRODIS -- a speech database and a phoneme-based language model for the study of predictability effects in Polish

The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text

On Retrieval Augmentation and the Limitations of Language Model Training

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

DailyMAE: Towards Pretraining Masked Autoencoders in One Day

Ethos: Rectifying Language Models in Orthogonal Parameter Space

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

Noise-Robust Keyword Spotting through Self-supervised Pretraining

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers

ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment

Bridging the Gap between Chemical Reaction Pretraining and Conditional Molecule Generation with a Unified Model

Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining

ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models

Finding Foundation Models for Time Series Classification with a PreText Task

United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once

Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models

CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals

Towards Cross-Table Masked Pretraining for Web Data Mining