语言大模型 / LLM - arXivDaily 专题

2306.14846 2023-10-25 cs.RO cs.CV cs.LG 79%

ViNT: A Foundation Model for Visual Navigation

Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyle Stachowicz, Kevin Black, Noriaki Hirose, Sergey Levine

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Accepted for oral presentation at CoRL 2023

2310.14840 2023-10-24 cs.CL 79%

Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution

Jaap Jumelet, Willem Zuidema

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP Findings 2023

2310.10567 2023-10-24 cs.CL 79%

RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling

Jingcheng Deng, Liang Pang, Huawei Shen, Xueqi Cheng

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to the Findings of EMNLP 2023

2310.14050 2023-10-24 cs.CL 79%

Code-Switching with Word Senses for Pretraining in Neural Machine Translation

Vivek Iyer, Edoardo Barba, Alexandra Birch, Jeff Z. Pan, Roberto Navigli

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP (Findings) 2023 Long Paper

2304.11389 2023-10-24 cs.CL 79%

Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens

Byung-Doh Oh, William Schuler

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of the Association for Computational Linguistics: EMNLP 2023

2310.13286 2023-10-23 cs.IR cs.AI 79%

Unified Pretraining for Recommendation via Task Hypergraphs

Mingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu, Chen Wang, Hao Peng, Philip S. Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by WSDM 2024

2310.07871 2023-10-23 cs.AI 79%

Hierarchical Pretraining on Multimodal Electronic Health Records

Xiaochen Wang, Junyu Luo, Jiaqi Wang, Ziyi Yin, Suhan Cui, Yuan Zhong, Yaqing Wang, Fenglong Ma

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by EMNLP 2023

2305.14281 2023-10-20 cs.CL cs.CV 79%

Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining

Emanuele Bugliarello, Aida Nematzadeh, Lisa Anne Hendricks

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP 2023

2305.14838 2023-10-17 cs.CL cs.SD eess.AS 79%

ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation

Chenyang Le, Yao Qian, Long Zhou, Shujie Liu, Yanmin Qian, Michael Zeng, Xuedong Huang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments NeurIPS 2023, Poster

2212.04755 2023-10-17 cs.CL 79%

From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader

Weiwen Xu, Xin Li, Wenxuan Zhang, Meng Zhou, Wai Lam, Luo Si, Lidong Bing

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to NeurIPS 2023

2308.12966 2023-10-16 cs.CV cs.CL 79%

Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, Jingren Zhou

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Code, demo and models are available at https://github.com/QwenLM/Qwen-VL

2307.15933 2023-10-16 cs.CL 79%

GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning

Soumyadeep Roy, Jonas Wallat, Sowmya S Sundaram, Wolfgang Nejdl, Niloy Ganguly

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 12 pages including appendix. Accepted for publication at 26th European Conference on Artificial Intelligence ECAI 2023

Journal ref Frontiers in Artificial Intelligence and Applications, Volume 372: ECAI 2023

2303.16887 2023-10-06 cs.CV cs.LG 79%

Towards Understanding the Effect of Pretraining Label Granularity

Guan Zhe Hong, Yin Cui, Ariel Fuxman, Stanley H. Chan, Enming Luo

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2309.17123 2023-10-02 cs.CV cs.LG 79%

Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining

Tianyu Han, Laura Žigutytė, Luisa Huck, Marc Huppertz, Robert Siepmann, Yossi Gandelsman, Christian Blüthgen, Firas Khader, Christiane Kuhl, Sven Nebelung, Jakob Kather, Daniel Truhn

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2211.05750 2023-09-26 cs.CL 79%

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Xiang Fan, Yiwei Lyu, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL Findings 2023

2309.08520 2023-09-18 cs.LG 79%

Scaling Laws for Sparsely-Connected Foundation Models

Elias Frantar, Carlos Riquelme, Neil Houlsby, Dan Alistarh, Utku Evci

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2308.09308 2023-09-18 cs.IR cs.CL 79%

Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification

Chenyu Zhao, Yunjiang Jiang, Yiming Qiu, Han Zhang, Wen-Yun Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 2 figures; accepted by CIKM2023

2308.15670 2023-09-06 cs.CV cs.AI 79%

Multimodal Foundation Models For Echocardiogram Interpretation

Matthew Christensen, Milos Vukadinovic, Neal Yuan, David Ouyang

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2308.14050 2023-08-29 cs.CV cs.AI 79%

PECon: Contrastive Pretraining to Enhance Feature Alignment between CT and EHR Data for Improved Pulmonary Embolism Diagnosis

Santosh Sanjeev, Salwa K. Al Khatib, Mai A. Shaaban, Ibrahim Almakky, Vijay Ram Papineni, Mohammad Yaqub

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2308.08129 2023-08-17 cs.LG q-bio.QM 79%

Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?

Shun Takashige, Masatoshi Hanai, Toyotaro Suzumura, Limin Wang, Kenjiro Taura

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2308.07146 2023-08-15 cs.CV cs.AI cs.MM 79%

CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation

Hongguang Zhu, Yunchao Wei, Xiaodan Liang, Chunjie Zhang, Yao Zhao

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by ICCV 2023. Code: https://github.com/KevinLight831/CTP

2308.04551 2023-08-10 eess.IV cs.CV cs.LG 79%

Improving Medical Image Classification in Noisy Labels Using Only Self-supervised Pretraining

Bidur Khanal, Binod Bhattarai, Bishesh Khanal, Cristian A. Linte

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted at MICCAI 2023 DEMI Workshop

2307.10457 2023-08-09 cs.CL 79%

Improving the Reusability of Pre-trained Language Models in Real-world Applications

Somayeh Ghanbarzadeh, Hamid Palangi, Yan Huang, Radames Cruz Moreno, Hamed Khanpour

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted as a long paper and awarded as the BEST Resaerch Paper in IEEE IRI'23 (IEEE 24th International conference on Information Reuse and Integrationfor Data Science)

2307.01759 2023-08-08 cs.CV cs.LG eess.IV 79%

Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification

Lucas Mahler, Qi Wang, Julius Steiglechner, Florian Birk, Samuel Heczko, Klaus Scheffler, Gabriele Lohmann

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.00083 2023-08-02 cs.CL cs.IR 79%

In-Context Retrieval-Augmented Language Models

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication in Transactions of the Association for Computational Linguistics (TACL). pre-MIT Press publication version

2307.13560 2023-08-01 cs.CL 79%

XDLM: Cross-lingual Diffusion Language Model for Machine Translation

Linyao Chen, Aosong Feng, Boming Yang, Zihui Li

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.CL

2106.11483 2023-07-27 cs.CL 79%

A Comprehensive Comparison of Pre-training Language Models

Tong Guo

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2209.10901 2023-07-20 cs.LG 79%

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

Manuel Goulão, Arlindo L. Oliveira

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2307.07487 2023-07-17 cs.CV cs.LG 79%

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Seung Wook Kim, Karsten Kreis, Antonio Torralba, Sanja Fidler

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Project page: https://research.nvidia.com/labs/toronto-ai/DreamTeacher/

2306.17649 2023-07-11 cs.CL 79%

Biomedical Language Models are Robust to Sub-optimal Tokenization

Bernal Jiménez Gutiérrez, Huan Sun, Yu Su

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments BioNLP @ ACL 2023

1. 预训练与数据 12393 篇

ViNT: A Foundation Model for Visual Navigation

Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution

RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling

Code-Switching with Word Senses for Pretraining in Neural Machine Translation

Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens

Unified Pretraining for Recommendation via Task Hypergraphs

Hierarchical Pretraining on Multimodal Electronic Health Records

Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining

ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation

From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader

Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning

Towards Understanding the Effect of Pretraining Label Granularity

Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Scaling Laws for Sparsely-Connected Foundation Models

Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification

Multimodal Foundation Models For Echocardiogram Interpretation

PECon: Contrastive Pretraining to Enhance Feature Alignment between CT and EHR Data for Improved Pulmonary Embolism Diagnosis

Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?

CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation

Improving Medical Image Classification in Noisy Labels Using Only Self-supervised Pretraining

Improving the Reusability of Pre-trained Language Models in Real-world Applications

Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification

In-Context Retrieval-Augmented Language Models

XDLM: Cross-lingual Diffusion Language Model for Machine Translation

A Comprehensive Comparison of Pre-training Language Models

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

Biomedical Language Models are Robust to Sub-optimal Tokenization