语言大模型 / LLM - arXivDaily 专题

2310.15819 2024-06-18 cs.CL cs.CY 83%

Generative Language Models Exhibit Social Identity Biases

Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden, Jon Roozenbeek

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments supplementary material, data, and code see https://osf.io/9ht32/?view_only=f0ab4b23325f4c31ad3e12a7353b55f5

2406.10721 2024-06-18 cs.RO cs.AI cs.CV 83%

RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics

Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, Dieter Fox

专题命中预训练与数据：language model(title,abstract)；prompting(abstract)；分类 cs.AI

2402.14652 2024-06-18 cs.CL 83%

Cleaner Pretraining Corpus Curation with Neural Web Scraping

Zhipeng Xu, Zhenghao Liu, Yukun Yan, Zhiyuan Liu, Ge Yu, Chenyan Xiong

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

2404.14027 2024-06-13 cs.CV cs.LG 83%

OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks

Sophia Sirko-Galouchenko, Alexandre Boulch, Spyros Gidaris, Andrei Bursuc, Antonin Vobecky, Patrick Pérez, Renaud Marlet

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.LG

Comments Accepted to CVPR 2024, Workshop on Autonomous Driving

2405.11093 2024-06-10 eess.AS cs.CL cs.MM cs.SD 83%

AudioSetMix: Enhancing Audio-Language Datasets with LLM-Assisted Augmentations

David Xu

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments typos corrected

2406.03872 2024-06-07 cs.CL cs.SD eess.AS 83%

BLSP-Emo: Towards Empathetic Large Speech-Language Models

Chen Wang, Minpeng Liao, Zhongqiang Huang, Junhong Wu, Chengqing Zong, Jiajun Zhang

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2406.03686 2024-06-07 cs.LG 83%

BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning

Artem Zholus, Maksim Kuznetsov, Roman Schutski, Rim Shayakhmetov, Daniil Polykovskiy, Sarath Chandar, Alex Zhavoronkov

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.LG

2403.04822 2024-05-28 cs.CV cs.LG 83%

UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining

ShengYun Peng, Aishwarya Chakravarthy, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.LG

2405.09250 2024-05-16 cs.CL 83%

New Textual Corpora for Serbian Language Modeling

Mihailo Škorić, Nikola Janković

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2404.19319 2024-05-01 cs.CL 83%

Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget

Minh Duc Bui, Fabian David Schmidt, Goran Glavaš, Katharina von der Wense

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to the 5th Workshop on Insights from Negative Results in NLP at NAACL 2024

2402.01176 2024-04-23 cs.CL cs.IR 83%

CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks

Xiaoxi Li, Zhicheng Dou, Yujia Zhou, Fangchao Liu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2404.11706 2024-04-19 cs.AI 83%

Pretraining Billion-scale Geospatial Foundational Models on Frontier

Aristeidis Tsaris, Philipe Ambrozio Dias, Abhishek Potnis, Junqi Yin, Feiyi Wang, Dalton Lunga

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI

2404.10939 2024-04-18 cs.CL 83%

More Room for Language: Investigating the Effect of Retrieval on Language Models

David Samuel, Lucas Georges Gabriel Charpentier, Sondre Wold

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments NAACL 2024

2404.03086 2024-04-05 stat.AP cs.CL 83%

Auditing the Use of Language Models to Guide Hiring Decisions

Johann D. Gaebler, Sharad Goel, Aziz Huq, Prasanna Tambe

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2404.00225 2024-04-02 cs.LG 83%

Heterogeneous Contrastive Learning for Foundation Models and Beyond

Lecheng Zheng, Baoyu Jing, Zihao Li, Hanghang Tong, Jingrui He

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

2311.08849 2024-03-26 cs.CL 83%

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

Yihong Liu, Peiqin Lin, Mingyang Wang, Hinrich Schütze

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments NAACL 2024 Findings

2403.13666 2024-03-21 cs.CL 83%

Grounding Spatial Relations in Text-Only Language Models

Gorka Azkune, Ander Salaberria, Eneko Agirre

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in Neural Networks

2403.08046 2024-03-14 cs.CL 83%

Big City Bias: Evaluating the Impact of Metropolitan Size on Computational Job Market Abilities of Language Models

Charlie Campanella, Rob van der Goot

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 5 pages, 3 figures, 2 tables, NLP4HR Workshop @ EACL 2024

2403.08032 2024-03-14 cs.CV cs.AI 83%

LG-Traj: LLM Guided Pedestrian Trajectory Prediction

Pranav Singh Chib, Pravendra Singh

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Under Review

2403.06965 2024-03-12 cs.CL 83%

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena

Leonie Weissweiler, Abdullatif Köksal, Hinrich Schütze

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2402.13374 2024-02-22 cs.CL 83%

Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems

Ivan Sekulić, Silvia Terragni, Victor Guimarães, Nghia Khau, Bruna Guedes, Modestas Filipavicius, André Ferreira Manso, Roland Mathis

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2401.15896 2024-02-06 cs.CV cs.AI 83%

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Qingpei Guo, Furong Xu, Hanxiao Zhang, Wang Ren, Ziping Ma, Lin Ju, Jian Wang, Jingdong Chen, Ming Yang

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI

2401.13565 2024-02-06 cs.CL 83%

Large Malaysian Language Model Based on Mistral for Enhanced Local Language Understanding

Husein Zolkepli, Aisyah Razak, Kamarul Adha, Ariff Nazhan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2401.16380 2024-01-30 cs.CL 83%

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, Navdeep Jaitly

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2203.08565 2024-01-30 cs.CL 83%

Geographic Adaptation of Pretrained Language Models

Valentin Hofmann, Goran Glavaš, Nikola Ljubešić, Janet B. Pierrehumbert, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments TACL 2024 (pre-MIT Press publication version)

2205.14912 2024-01-10 cs.CL 83%

E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by IEEE TKDE 2023

2305.00090 2024-01-08 cs.CL 83%

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schütze

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

2312.05435 2023-12-12 cs.CL 83%

Enhancing Robustness of Foundation Model Representations under Provenance-related Distribution Shifts

Xiruo Ding, Zhecheng Sheng, Brian Hur, Feng Chen, Serguei V. S. Pakhomov, Trevor Cohen

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in Workshop on Distribution Shifts, 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

2310.20138 2023-12-06 cs.CR cs.CL 83%

DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models

Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, Deyi Xiong

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments EMNLP 2023

2311.15436 2023-11-28 cs.CL 83%

Learning to Skip for Language Modeling

Dewen Zeng, Nan Du, Tao Wang, Yuanzhong Xu, Tao Lei, Zhifeng Chen, Claire Cui

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1. 预训练与数据 12393 篇

Generative Language Models Exhibit Social Identity Biases

RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics

Cleaner Pretraining Corpus Curation with Neural Web Scraping

OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks

AudioSetMix: Enhancing Audio-Language Datasets with LLM-Assisted Augmentations

BLSP-Emo: Towards Empathetic Large Speech-Language Models

BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning

UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining

New Textual Corpora for Serbian Language Modeling

Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget

CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks

Pretraining Billion-scale Geospatial Foundational Models on Frontier

More Room for Language: Investigating the Effect of Retrieval on Language Models

Auditing the Use of Language Models to Guide Hiring Decisions

Heterogeneous Contrastive Learning for Foundation Models and Beyond

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

Grounding Spatial Relations in Text-Only Language Models

Big City Bias: Evaluating the Impact of Metropolitan Size on Computational Job Market Abilities of Language Models

LG-Traj: LLM Guided Pedestrian Trajectory Prediction

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena

Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Large Malaysian Language Model Based on Mistral for Enhanced Local Language Understanding

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

Geographic Adaptation of Pretrained Language Models

E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

Enhancing Robustness of Foundation Model Representations under Provenance-related Distribution Shifts

DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models

Learning to Skip for Language Modeling