语言大模型 / LLM - arXivDaily 专题

2401.03735 2024-11-15 cs.CL 83%

Language Models Encode the Value of Numbers Linearly

Fangwei Zhu, Damai Dai, Zhifang Sui

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments The code and data are available at https://github.com/solitaryzero/NumProbe

2411.05489 2024-11-11 cs.LG cs.CV 83%

Do Histopathological Foundation Models Eliminate Batch Effects? A Comparative Study

Jonah Kömen, Hannah Marienwald, Jonas Dippel, Julius Hense

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments Accepted to AIM-FM Workshop @ NeurIPS'24

2411.04642 2024-11-08 cs.CV cs.AI 83%

TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models

Jonathan Fhima, Elad Ben Avraham, Oren Nuriel, Yair Kittenplon, Roy Ganz, Aviad Aberdam, Ron Litman

专题命中预训练与数据：language model(title)；LLM(abstract)；pretraining(abstract)；分类 cs.AI

2410.19965 2024-10-29 cs.CV cs.AI cs.DC 83%

OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery

Philipe Dias, Aristeidis Tsaris, Jordan Bowman, Abhishek Potnis, Jacob Arndt, H. Lexie Yang, Dalton Lunga

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI

Journal ref The 32nd ACM International Conference on Advances in Geographic Information Systems (SIGSPATIAL 24), October 29-November 1, 2024, Atlanta, GA, USA. ACM, New York, NY, USA, 4 pages

2410.19290 2024-10-28 cs.CL 83%

Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning

Yujian Liu, Shiyu Chang, Tommi Jaakkola, Yang Zhang

专题命中预训练与数据：LLM(title,abstract)；SFT(abstract)；分类 cs.CL

2410.16424 2024-10-23 cs.LG 83%

Promoting cross-modal representations to improve multimodal foundation models for physiological signals

Ching Fang, Christopher Sandino, Behrooz Mahasseni, Juri Minxha, Hadi Pouransari, Erdrin Azemi, Ali Moin, Ellen Zippi

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments NeurIPS 2024 AIM-FM Workshop

2407.06380 2024-10-22 cs.CL 83%

Data, Data Everywhere: A Guide for Pretraining Dataset Construction

Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted as an oral presentation at EMNLP 2024

2404.12897 2024-10-18 cs.CL 83%

Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning

Ahmed Elshabrawy, Yongxin Huang, Iryna Gurevych, Alham Fikri Aji

专题命中预训练与数据：prompting(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2402.17302 2024-10-08 cs.CL 83%

Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese

Rifki Afina Putri, Faiz Ghifari Haznitrama, Dea Adhista, Alice Oh

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2024 Camera-Ready

2410.02271 2024-10-04 cs.SD cs.AI eess.AS 83%

CoLLAP: Contrastive Long-form Language-Audio Pretraining with Musical Temporal Structure Augmentation

Junda Wu, Warren Li, Zachary Novack, Amit Namburi, Carol Chen, Julian McAuley

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.AI

Comments 4 pages

2409.19117 2024-10-01 cs.LG eess.SP 83%

Range-aware Positional Encoding via High-order Pretraining: Theory and Practice

Viet Anh Nguyen, Nhat Khang Ngo, Truong Son Hy

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.LG

2409.15626 2024-09-25 cs.IR cs.CL 83%

Qualitative Insights Tool (QualIT): LLM Enhanced Topic Modeling

Satya Kapoor, Alex Gil, Sreyoshi Bhaduri, Anshul Mittal, Rutu Mulkar

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 6 pages, 4 tables, 1 figure

2409.05247 2024-09-10 cs.CL 83%

Socially Responsible Data for Large Multilingual Language Models

Andrew Smart, Ben Hutchinson, Lameck Mbangula Amugongo, Suzanne Dikker, Alex Zito, Amber Ebinama, Zara Wudiri, Ding Wang, Erin van Liemt, João Sedoc, Seyi Olojo, Stanley Uwakwe, Edem Wornyo, Sonja Schmer-Galunder, Jamila Smith-Loud

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Journal ref ACM conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 2024

2408.16173 2024-08-30 cs.DB cs.AI 83%

LLM-assisted Labeling Function Generation for Semantic Type Detection

Chenjie Li, Dan Zhang, Jin Wang

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments VLDB'24-DATAI

2311.02333 2024-08-26 cs.LG q-bio.GN 83%

Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision

Aditya Malusare, Harish Kothandaraman, Dipesh Tamboli, Nadia A. Lanman, Vaneet Aggarwal

专题命中预训练与数据：foundation model(title,abstract)；language model(abstract)；分类 cs.LG

Comments Accepted to OUP Bioinformatics Advances

2405.14277 2024-08-08 cs.CL 83%

Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysis

Sabri Boughorbel, MD Rizwan Parvez, Majd Hawasly

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 16 pages

2402.16420 2024-08-07 cs.CL 83%

Predicting Sustainable Development Goals Using Course Descriptions -- from LLMs to Conventional Foundation Models

Lev Kharlashkin, Melany Macias, Leo Huovinen, Mika Hämäläinen

专题命中预训练与数据：foundation model(title)；LLM(abstract)；language model(abstract)；分类 cs.CL

Comments 3 figures, 2 tables

Journal ref Journal of Data Mining & Digital Humanities, NLP4DH (April 29, 2024) jdmdh:13127

2309.10931 2024-08-05 cs.CL 83%

A Family of Pretrained Transformer Language Models for Russian

Dmitry Zmitrovich, Alexander Abramov, Andrey Kalmykov, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Vitalii Kadulin, Sergey Markov, Tatiana Shavrina, Vladislav Mikhailov, Alena Fenogenova

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments LREC-COLING-2024

Journal ref https://aclanthology.org/2024.lrec-main.45/

2408.00932 2024-08-05 cs.CV cs.CL 83%

Towards Zero-Shot Annotation of the Built Environment with Vision-Language Models (Vision Paper)

Bin Han, Yiwei Yang, Anat Caspi, Bill Howe

专题命中预训练与数据：language model(title,abstract)；prompting(abstract)；分类 cs.CL

2407.15641 2024-07-23 eess.AS cs.LG cs.SD 83%

Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Models

Shahan Nercessian, Johannes Imort, Ninon Devis, Frederik Blang

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments 8 pages, 2 figures. Accepted to the 25th Conference of the International Society for Music Information Retrieval (ISMIR)

2403.16497 2024-07-16 cs.CV cs.LG 83%

PathoTune: Adapting Visual Foundation Model to Pathological Specialists

Jiaxuan Lu, Fang Yan, Xiaofan Zhang, Yue Gao, Shaoting Zhang

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments MICCAI 2024

2305.14321 2024-07-11 cs.CL 83%

ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings

William Brannon, Wonjune Kang, Suyash Fulay, Hang Jiang, Brandon Roy, Deb Roy, Jad Kabbara

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments New visualizations, added references, and an application to community detection. To appear at the TextGraphs workshop @ ACL 2024. 21 pages, 5 figures, 13 tables

2305.13684 2024-07-08 cs.CL 83%

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models

Peiqin Lin, Chengzhi Hu, Zheyu Zhang, André F. T. Martins, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments EACL 2024 Findings

2402.04615 2024-07-08 cs.CV cs.AI 83%

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor Cărbune, Jason Lin, Jindong Chen, Abhanshu Sharma

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments Accepted to International Joint Conference on Artificial Intelligence (IJCAI), 2024. Revision Notes: full version of the paper, including 1) Camera-ready version for IJCAI-24; 2) Appendices that are mentioned, but not included in 1)

2210.09475 2024-07-03 cs.LG 83%

FIMP: Foundation Model-Informed Message Passing for Graph Neural Networks

Syed Asad Rizvi, Nazreen Pallikkavaliyaveetil, David Zhang, Zhuoyang Lyu, Nhi Nguyen, Haoran Lyu, Benjamin Christensen, Josue Ortega Caro, Antonio H. O. Fonseca, Emanuele Zappala, Maryam Bagherian, Christopher Averill, Chadi G. Abdallah, Amin Karbasi, Rex Ying, Maria Brbic, Rahul Madhav Dhodapkar, David van Dijk

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments 16 pages (12 + 4 pages appendix). 5 figures and 4 tables

2406.18085 2024-06-27 cs.CL 83%

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints

Ran Song, Shizhu He, Shengxiang Gao, Li Cai, Kang Liu, Zhengtao Yu, Jun Zhao

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments 11 pages, ACL 2023

2406.16690 2024-06-25 cs.CL 83%

Scaling Laws for Linear Complexity Language Models

Xuyang Shen, Dong Li, Ruitao Leng, Zhen Qin, Weigao Sun, Yiran Zhong

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Technical report. Yiran Zhong is the corresponding author

2406.13578 2024-06-21 cs.CL 83%

Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration

Han-Cheng Yu, Yu-An Shih, Kin-Man Law, Kai-Yu Hsieh, Yu-Chen Cheng, Hsin-Chih Ho, Zih-An Lin, Wen-Chuan Hsu, Yao-Chung Fan

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Findings at ACL 2024

2406.11938 2024-06-19 cs.AI cs.MA 83%

Tracking the perspectives of interacting language models

Hayden Helm, Brandon Duderstadt, Youngser Park, Carey E. Priebe

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI

2404.04633 2024-06-18 cs.CL 83%

Context versus Prior Knowledge in Language Models

Kevin Du, Vésteinn Snæbjarnarson, Niklas Stoehr, Jennifer C. White, Aaron Schein, Ryan Cotterell

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Long paper accepted at ACL 2024

1. 预训练与数据 12393 篇

Language Models Encode the Value of Numbers Linearly

Do Histopathological Foundation Models Eliminate Batch Effects? A Comparative Study

TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models

OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery

Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning

Promoting cross-modal representations to improve multimodal foundation models for physiological signals

Data, Data Everywhere: A Guide for Pretraining Dataset Construction

Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning

Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese

CoLLAP: Contrastive Long-form Language-Audio Pretraining with Musical Temporal Structure Augmentation

Range-aware Positional Encoding via High-order Pretraining: Theory and Practice

Qualitative Insights Tool (QualIT): LLM Enhanced Topic Modeling

Socially Responsible Data for Large Multilingual Language Models

LLM-assisted Labeling Function Generation for Semantic Type Detection

Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level Precision

Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysis

Predicting Sustainable Development Goals Using Course Descriptions -- from LLMs to Conventional Foundation Models

A Family of Pretrained Transformer Language Models for Russian

Towards Zero-Shot Annotation of the Built Environment with Vision-Language Models (Vision Paper)

Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Models

PathoTune: Adapting Visual Foundation Model to Pathological Specialists

ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings

mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

FIMP: Foundation Model-Informed Message Passing for Graph Neural Networks

Multilingual Knowledge Graph Completion from Pretrained Language Models with Knowledge Constraints

Scaling Laws for Linear Complexity Language Models

Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration

Tracking the perspectives of interacting language models

Context versus Prior Knowledge in Language Models