语言大模型 / LLM - arXivDaily 专题

2401.10440 2024-10-10 cs.CL 79%

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah A. Smith, Luke Zettlemoyer

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2024

2409.18082 2024-10-08 cs.RO cs.AI cs.CV 79%

SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation

Xin Li, Siyuan Huang, Qiaojun Yu, Zhengkai Jiang, Ce Hao, Yimeng Zhu, Hongsheng Li, Peng Gao, Cewu Lu

专题命中预训练与数据：language model(title,abstract)；分类 cs.AI

2210.11815 2024-10-08 cs.CV cs.LG 79%

Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection

Jules BOURCIER, Thomas Floquet, Gohar Dashyan, Tugdual Ceillier, Karteek Alahari, Jocelyn Chanussot

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Journal ref Conference on Artificial Intelligence for Defense (CAID) 2022, DGA Maîtrise de l'Information, Nov 2022, Rennes, France

2410.03194 2024-10-07 cs.CL 79%

Parallel Corpus Augmentation using Masked Language Models

Vibhuti Kumari, Narayana Murthy Kavi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 21 Pages, 3 Figures. arXiv admin note: text overlap with arXiv:2011.01536 by other authors

2410.03001 2024-10-07 cs.CL 79%

Can Transformers Learn $n$-gram Language Models?

Anej Svete, Nadav Borenstein, Mike Zhou, Isabelle Augenstein, Ryan Cotterell

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2406.08473 2024-10-03 cs.LG 79%

Strategies for Pretraining Neural Operators

Anthony Zhou, Cooper Lorsung, AmirPouya Hemmasian, Amir Barati Farimani

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 29 pages, 5 figures

Journal ref Transactions on Machine Learning Research, 2024

2409.19735 2024-10-01 cs.CL 79%

Scrambled text: training Language Models to correct OCR errors using synthetic data

Jonathan Bourne

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 21 pages, 6300 words, 6 Figures, 5 tables

2409.18100 2024-09-27 cs.CV cs.LG 79%

Self-supervised Pretraining for Cardiovascular Magnetic Resonance Cine Segmentation

Rob A. J. de Mooij, Josien P. W. Pluim, Cian M. Scannell

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted to Data Engineering in Medical Imaging (DEMI) Workshop at MICCAI 2024

2409.18044 2024-09-27 cs.CL 79%

Unveiling the Role of Pretraining in Direct Speech Translation

Belen Alastruey, Gerard I. Gállego, Marta R. Costa-jussà

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP 2024

2409.15486 2024-09-25 cs.CV cs.AI 79%

VLMine: Long-Tail Data Mining with Vision Language Models

Mao Ye, Gregory P. Meyer, Zaiwei Zhang, Dennis Park, Siva Karthik Mustikovela, Yuning Chai, Eric M Wolff

专题命中预训练与数据：language model(title,abstract)；分类 cs.AI

2405.00117 2024-09-24 cs.CV cs.AI 79%

Training a high-performance retinal foundation model with half-the-data and 400 times less compute

Justin Engelmann, Miguel O. Bernabeu

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2409.09811 2024-09-17 cs.LG cs.NA math.NA physics.flu-dyn 79%

PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics

Yuxuan Liu, Jingmin Sun, Xinjie He, Griffin Pinney, Zecheng Zhang, Hayden Schaeffer

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2409.03701 2024-09-11 cs.CL cs.SD eess.AS 79%

LAST: Language Model Aware Speech Tokenization

Arnon Turetzky, Yossi Adi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2409.05162 2024-09-10 cs.CV cs.LG 79%

Can OOD Object Detectors Learn from Foundation Models?

Jiahui Liu, Xin Wen, Shizhen Zhao, Yingxian Chen, Xiaojuan Qi

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments 19 pages, 4 figures

Journal ref European Conference on Computer Vision (ECCV) 2024

2409.03299 2024-09-06 cs.RO cs.LG 79%

Bringing the RT-1-X Foundation Model to a SCARA robot

Jonathan Salzer, Arnoud Visser

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments 14 pages, submitted to the joint Artificial Intelligence & Machine Learning conference for Belgium, Netherlands & Luxembourg (BNAIC/BeNeLearn)

2406.08800 2024-08-30 cs.SD cs.LG eess.AS 79%

Can Synthetic Audio From Generative Foundation Models Assist Audio Recognition and Speech Modeling?

Tiantian Feng, Dimitrios Dimitriadis, Shrikanth Narayanan

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Accepted to 2024 INTERSPEECH; corrections to ActivityNet labels

2408.15251 2024-08-29 cs.CV cs.LG 79%

TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferability

Yan Lin, Tonglong Wei, Zeyu Zhou, Haomin Wen, Jilin Hu, Shengnan Guo, Youfang Lin, Huaiyu Wan

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2305.14681 2024-08-27 cs.CL 79%

Emergent inabilities? Inverse scaling over the course of pretraining

James A. Michaelov, Benjamin K. Bergen

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments Accepted to Findings of EMNLP 2023

2405.01040 2024-08-16 cs.CV cs.CL eess.IV 79%

Few Shot Class Incremental Learning using Vision-Language models

Anurag Kumar, Chinmay Bharti, Saikat Dutta, Srikrishna Karanam, Biplab Banerjee

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2404.17143 2024-08-16 cs.CL 79%

Quantifying Memorization and Detecting Training Data of Pre-trained Language Models using Japanese Newspaper

Shotaro Ishihara, Hiromu Takahashi

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments The 17th International Natural Language Generation Conference

2408.07869 2024-08-16 cs.LG 79%

A Systematic Evaluation of Generated Time Series and Their Effects in Self-Supervised Pretraining

Audrey Der, Chin-Chia Michael Yeh, Xin Dai, Huiyuan Chen, Yan Zheng, Yujie Fan, Zhongfang Zhuang, Vivian Lai, Junpeng Wang, Liang Wang, Wei Zhang, Eamonn Keogh

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments To appear in CIKM 2024 as a short paper; the version here is the self-contained version that includes the non-mandatory supplementary material available on the paper's companion website

2408.00640 2024-08-16 eess.IV cs.AI cs.CV 79%

AMAES: Augmented Masked Autoencoder Pretraining on Public Brain MRI Data for 3D-Native Segmentation

Asbjørn Munk, Jakob Ambsdorf, Sebastian Llambias, Mads Nielsen

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted at ADSMI @ MICCAI 2024

2408.07106 2024-08-15 cs.SE cs.CR cs.LG 79%

"You still have to study" -- On the Security of LLM generated code

Stefan Goetz, Andreas Schaad

专题命中预训练与数据：LLM(title)；prompting(abstract)；分类 cs.LG

2408.05772 2024-08-13 cs.CV cs.AI 79%

An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set

Chaoyi Ai

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2407.17638 2024-08-13 cs.CL 79%

Time Matters: Examine Temporal Effects on Biomedical Language Models

Weisi Liu, Zhe He, Xiaolei Huang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to AMIA 2024 Annual Symposium

2408.03433 2024-08-08 cs.CV cs.LG 79%

Hybrid diffusion models: combining supervised and generative pretraining for label-efficient fine-tuning of segmentation models

Bruno Sauvalle, Mathieu Salzmann

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 19 pages

2408.02859 2024-08-07 eess.IV cs.AI cs.CV 79%

Multistain Pretraining for Slide Representation Learning in Pathology

Guillaume Jaume, Anurag Vaidya, Andrew Zhang, Andrew H. Song, Richard J. Chen, Sharifa Sahai, Dandan Mo, Emilio Madrigal, Long Phi Le, Faisal Mahmood

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments ECCV'24

2408.02244 2024-08-06 cs.CV cs.AI 79%

Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets

Lucas Choi, Ross Greer

专题命中预训练与数据：language model(title)；foundation model(abstract)；分类 cs.AI

2207.00982 2024-08-06 q-bio.QM cs.LG 79%

Linguistically inspired roadmap for building biologically reliable protein language models

Mai Ha Vu, Rahmad Akbar, Philippe A. Robert, Bartlomiej Swiatczak, Victor Greiff, Geir Kjetil Sandve, Dag Trygve Truslew Haug

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

Comments 27 pages, 4 figures

Journal ref Nat Mach Intell (2023)

2401.06443 2024-08-01 cs.CL 79%

BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining

Minjun Kim, Seungwoo Song, Youhan Lee, Haneol Jang, Kyungtae Lim

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

1. 预训练与数据 12393 篇

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation

Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection

Parallel Corpus Augmentation using Masked Language Models

Can Transformers Learn $n$-gram Language Models?

Strategies for Pretraining Neural Operators

Scrambled text: training Language Models to correct OCR errors using synthetic data

Self-supervised Pretraining for Cardiovascular Magnetic Resonance Cine Segmentation

Unveiling the Role of Pretraining in Direct Speech Translation

VLMine: Long-Tail Data Mining with Vision Language Models

Training a high-performance retinal foundation model with half-the-data and 400 times less compute

PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics

LAST: Language Model Aware Speech Tokenization

Can OOD Object Detectors Learn from Foundation Models?

Bringing the RT-1-X Foundation Model to a SCARA robot

Can Synthetic Audio From Generative Foundation Models Assist Audio Recognition and Speech Modeling?

TrajFM: A Vehicle Trajectory Foundation Model for Region and Task Transferability

Emergent inabilities? Inverse scaling over the course of pretraining

Few Shot Class Incremental Learning using Vision-Language models

Quantifying Memorization and Detecting Training Data of Pre-trained Language Models using Japanese Newspaper

A Systematic Evaluation of Generated Time Series and Their Effects in Self-Supervised Pretraining

AMAES: Augmented Masked Autoencoder Pretraining on Public Brain MRI Data for 3D-Native Segmentation

"You still have to study" -- On the Security of LLM generated code

An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set

Time Matters: Examine Temporal Effects on Biomedical Language Models

Hybrid diffusion models: combining supervised and generative pretraining for label-efficient fine-tuning of segmentation models

Multistain Pretraining for Slide Representation Learning in Pathology

Evaluating Vision-Language Models for Zero-Shot Detection, Classification, and Association of Motorcycles, Passengers, and Helmets

Linguistically inspired roadmap for building biologically reliable protein language models

BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining