语言大模型 / LLM

Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification

Kush Dubey

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments To appear in the GenBench Workshop at EMNLP 2024

Target-Aware Language Modeling via Granular Data Sampling

Ernie Chang, Pin-Jie Lin, Yang Li, Changsheng Zhao, Daeil Kim, Rastislav Rabatin, Zechun Liu, Yangyang Shi, Vikas Chandra

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Main Conference, 9 pages, 6 figures, 3 tables

Towards a graph-based foundation model for network traffic analysis

Louis Van Langendonck, Ismael Castell-Uroz, Pere Barlet-Ros

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Pre-print of Accepted Workshop paper to 3rd GNNet, co-located with CoNEXT'24

Arctic-SnowCoder: Demystifying High-Quality Data in Code Pretraining

Yuxiang Wei, Hojae Han, Rajhans Samdani

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning

Jingyu Hu, Weiru Liu, Mengnan Du

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

A Billion-scale Foundation Model for Remote Sensing Images

Keumgang Cha, Junghoon Seo, Taekyung Lee

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments This manuscript is the accepted version for IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (IEEE J-STARS)

Exploration of Masked and Causal Language Modelling for Text Generation

Nicolo Micheletti, Samuel Belkadi, Lifeng Han, Goran Nenadic

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments working paper - under review

What Evidence Do Language Models Find Convincing?

Alexander Wan, Eric Wallace, Dan Klein

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments ACL 2024 (Main)

STaR-GATE: Teaching Language Models to Ask Clarifying Questions

Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman

专题命中 预训练与数据 ：language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Arithmetic with Language Models: from Memorization to Computation

Davide Maltoni, Matteo Ferrara

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments The article has been accepted for publication in Elsevier Neural Networks journal. The final version is available on the Elsevier ScienceDirect platform

Journal ref \Neural Networks, vol. 179, 2024

Learning to Plan for Language Modeling from Unlabeled Data

Nathan Cornille, Marie-Francine Moens, Florian Mai

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Published at COLM 2024

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models

Samuele Poppi, Tobia Poppi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments ECCV 2024

The Silent Curriculum: How Does LLM Monoculture Shape Educational Content and Its Accessibility?

Aman Priyanshu, Supriti Vijay

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 5 pages and 4 figures. Accepted at The Workshop on Global AI Cultures at the International Conference on Learning Representations, 2024 (ICLR'24)

Super Tiny Language Models

Dylan Hillier, Leon Guertler, Cheston Tan, Palaash Agrawal, Chen Ruirui, Bobby Cheng

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 4 figures

DE-COP: Detecting Copyrighted Content in Language Models Training Data

André V. Duarte, Xuandong Zhao, Arlindo L. Oliveira, Lei Li

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Exploring Spatial Representations in the Historical Lake District Texts with LLM-based Relation Extraction

Erum Haris, Anthony G. Cohn, John G. Stell

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Knowledge Fusion By Evolving Weights of Language Models

Guodong Du, Jing Li, Hanting Liu, Runhua Jiang, Shuyang Yu, Yifei Guo, Sim Kuan Goh, Ho-Kin Tang

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL2024 Findings

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale

Xiang Hu, Pengyu Ji, Qingyang Zhu, Wei Wu, Kewei Tu

专题命中 预训练与数据 ：language model(title,abstract)；SLM(abstract)；分类 cs.CL、cs.AI

Comments accepted by ACL 2024

Unlearning Traces the Influential Training Data of Language Models

Masaru Isonuma, Ivan Titov

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments 14 pages, to appear in ACL2024 main conference (long paper)

In-Context Unlearning: Language Models as Few Shot Unlearners

Martin Pawelczyk, Seth Neel, Himabindu Lakkaraju

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICML 2024

Balanced Data Sampling for Language Model Training with Clustering

Yunfan Shao, Linyang Li, Zhaoye Fei, Hang Yan, Dahua Lin, Xipeng Qiu

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments ACL 2024 (findings), Code is released at https://github.com/choosewhatulike/cluster-clip

DocReLM: Mastering Document Retrieval with Language Model

Gengchen Wei, Xinle Pang, Tianning Zhang, Yu Sun, Xun Qian, Chen Lin, Han-Sen Zhong, Wanli Ouyang

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

LMD3: Language Model Data Density Dependence

John Kirchenbauer, Garrett Honke, Gowthami Somepalli, Jonas Geiping, Daphne Ippolito, Katherine Lee, Tom Goldstein, David Andre

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 10 pages in the main body

Türkçe Dil Modellerinin Performans Karşılaştırması Performance Comparison of Turkish Language Models

Eren Dogan, M. Egemen Uzun, Atahan Uz, H. Emre Seyrek, Ahmed Zeer, Ezgi Sevi, H. Toprak Kesgin, M. Kaan Yuce, M. Fatih Amasyali

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments in Turkish language. Bazı çalışmaları içermediğini söyleyen hakem yorumu nedeniyle bir konferanstan kabul almadı. Ancak hakemin bahsettiği çalışmalar bildiri gönderme son tarihinde yayınlanmamıştı

Language Models Implement Simple Word2Vec-style Vector Arithmetic

Jack Merullo, Carsten Eickhoff, Ellie Pavlick

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments NAACL

Emergent Abilities in Reduced-Scale Generative Language Models

Sherin Muckatira, Vijeta Deshpande, Vladislav Lialin, Anna Rumshisky

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments 16 pages, 4 figures. Accepted to NAACL 2024 Findings

Bridging Remote Sensors with Multisensor Geospatial Foundation Models

Boran Han, Shuai Zhang, Xingjian Shi, Markus Reichstein

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accepted to CVPR

Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives

Gustave Cortal

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

A Knowledge-Injected Curriculum Pretraining Framework for Question Answering

Xin Lin, Tianhuang Su, Zhenya Huang, Shangzi Xue, Haifeng Liu, Enhong Chen

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by WWW 2024

On the Scaling Laws of Geographical Representation in Language Models

Nathan Godey, Éric de la Clergerie, Benoît Sagot

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at LREC-COLING 2024