语言大模型 / LLM - arXivDaily 专题

2406.03872 2024-06-07 cs.CL cs.SD eess.AS 83%

BLSP-Emo: Towards Empathetic Large Speech-Language Models

Chen Wang, Minpeng Liao, Zhongqiang Huang, Junhong Wu, Chengqing Zong, Jiajun Zhang

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2406.03686 2024-06-07 cs.LG 83%

BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning

Artem Zholus, Maksim Kuznetsov, Roman Schutski, Rim Shayakhmetov, Daniil Polykovskiy, Sarath Chandar, Alex Zhavoronkov

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.LG

2403.04822 2024-05-28 cs.CV cs.LG 83%

UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining

ShengYun Peng, Aishwarya Chakravarthy, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.LG

2405.09250 2024-05-16 cs.CL 83%

New Textual Corpora for Serbian Language Modeling

Mihailo Škorić, Nikola Janković

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2404.19319 2024-05-01 cs.CL 83%

Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget

Minh Duc Bui, Fabian David Schmidt, Goran Glavaš, Katharina von der Wense

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to the 5th Workshop on Insights from Negative Results in NLP at NAACL 2024

2402.01176 2024-04-23 cs.CL cs.IR 83%

CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks

Xiaoxi Li, Zhicheng Dou, Yujia Zhou, Fangchao Liu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2404.11706 2024-04-19 cs.AI 83%

Pretraining Billion-scale Geospatial Foundational Models on Frontier

Aristeidis Tsaris, Philipe Ambrozio Dias, Abhishek Potnis, Junqi Yin, Feiyi Wang, Dalton Lunga

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI

2404.10939 2024-04-18 cs.CL 83%

More Room for Language: Investigating the Effect of Retrieval on Language Models

David Samuel, Lucas Georges Gabriel Charpentier, Sondre Wold

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments NAACL 2024

2404.03086 2024-04-05 stat.AP cs.CL 83%

Auditing the Use of Language Models to Guide Hiring Decisions

Johann D. Gaebler, Sharad Goel, Aziz Huq, Prasanna Tambe

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2404.00225 2024-04-02 cs.LG 83%

Heterogeneous Contrastive Learning for Foundation Models and Beyond

Lecheng Zheng, Baoyu Jing, Zihao Li, Hanghang Tong, Jingrui He

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

2311.08849 2024-03-26 cs.CL 83%

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

Yihong Liu, Peiqin Lin, Mingyang Wang, Hinrich Schütze

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments NAACL 2024 Findings

2403.13666 2024-03-21 cs.CL 83%

Grounding Spatial Relations in Text-Only Language Models

Gorka Azkune, Ander Salaberria, Eneko Agirre

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in Neural Networks

2403.08046 2024-03-14 cs.CL 83%

Big City Bias: Evaluating the Impact of Metropolitan Size on Computational Job Market Abilities of Language Models

Charlie Campanella, Rob van der Goot

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 5 pages, 3 figures, 2 tables, NLP4HR Workshop @ EACL 2024

2403.08032 2024-03-14 cs.CV cs.AI 83%

LG-Traj: LLM Guided Pedestrian Trajectory Prediction

Pranav Singh Chib, Pravendra Singh

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Under Review

2403.06965 2024-03-12 cs.CL 83%

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena

Leonie Weissweiler, Abdullatif Köksal, Hinrich Schütze

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2402.13374 2024-02-22 cs.CL 83%

Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems

Ivan Sekulić, Silvia Terragni, Victor Guimarães, Nghia Khau, Bruna Guedes, Modestas Filipavicius, André Ferreira Manso, Roland Mathis

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2401.15896 2024-02-06 cs.CV cs.AI 83%

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Qingpei Guo, Furong Xu, Hanxiao Zhang, Wang Ren, Ziping Ma, Lin Ju, Jian Wang, Jingdong Chen, Ming Yang

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI

2401.13565 2024-02-06 cs.CL 83%

Large Malaysian Language Model Based on Mistral for Enhanced Local Language Understanding

Husein Zolkepli, Aisyah Razak, Kamarul Adha, Ariff Nazhan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2401.16380 2024-01-30 cs.CL 83%

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, Navdeep Jaitly

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2203.08565 2024-01-30 cs.CL 83%

Geographic Adaptation of Pretrained Language Models

Valentin Hofmann, Goran Glavaš, Nikola Ljubešić, Janet B. Pierrehumbert, Hinrich Schütze

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments TACL 2024 (pre-MIT Press publication version)

2205.14912 2024-01-10 cs.CL 83%

E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by IEEE TKDE 2023

2305.00090 2024-01-08 cs.CL 83%

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schütze

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

2312.05435 2023-12-12 cs.CL 83%

Enhancing Robustness of Foundation Model Representations under Provenance-related Distribution Shifts

Xiruo Ding, Zhecheng Sheng, Brian Hur, Feng Chen, Serguei V. S. Pakhomov, Trevor Cohen

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in Workshop on Distribution Shifts, 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

2310.20138 2023-12-06 cs.CR cs.CL 83%

DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models

Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, Deyi Xiong

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments EMNLP 2023

2311.15436 2023-11-28 cs.CL 83%

Learning to Skip for Language Modeling

Dewen Zeng, Nan Du, Tao Wang, Yuanzhong Xu, Tao Lei, Zhifeng Chen, Claire Cui

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2311.01307 2023-11-03 cs.CL 83%

The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models

Lovisa Hagström, Denitsa Saynova, Tobias Norlund, Moa Johansson, Richard Johansson

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2023

2311.00268 2023-11-02 cs.CL 83%

Syntactic Inductive Bias in Transformer Language Models: Especially Helpful for Low-Resource Languages?

Luke Gessler, Nathan Schneider

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted at CoNLL 2023

2305.14214 2023-10-24 cs.CL 83%

CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models

Benjamin Minixhofer, Jonas Pfeiffer, Ivan Vulić

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments EMNLP 2023

2310.12936 2023-10-24 cs.CL 83%

A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models

Yi Zhou, Jose Camacho-Collados, Danushka Bollegala

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023 main conference

2306.17525 2023-10-20 cond-mat.mtrl-sci cond-mat.dis-nn cond-mat.mes-hall cond-mat.other cs.AI 83%

MeLM, a generative pretrained language modeling framework that solves forward and inverse mechanics problems

Markus J. Buehler

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI

1. 预训练与数据 12379 篇

BLSP-Emo: Towards Empathetic Large Speech-Language Models

BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning

UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining

New Textual Corpora for Serbian Language Modeling

Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget

CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks

Pretraining Billion-scale Geospatial Foundational Models on Frontier

More Room for Language: Investigating the Effect of Retrieval on Language Models

Auditing the Use of Language Models to Guide Hiring Decisions

Heterogeneous Contrastive Learning for Foundation Models and Beyond

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

Grounding Spatial Relations in Text-Only Language Models

Big City Bias: Evaluating the Impact of Metropolitan Size on Computational Job Market Abilities of Language Models

LG-Traj: LLM Guided Pedestrian Trajectory Prediction

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena

Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Large Malaysian Language Model Based on Mistral for Enhanced Local Language Understanding

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

Geographic Adaptation of Pretrained Language Models

E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

Enhancing Robustness of Foundation Model Representations under Provenance-related Distribution Shifts

DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models

Learning to Skip for Language Modeling

The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models

Syntactic Inductive Bias in Transformer Language Models: Especially Helpful for Low-Resource Languages?

CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models

A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models

MeLM, a generative pretrained language modeling framework that solves forward and inverse mechanics problems