语言大模型 / LLM

AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization

Fabrycio Leite Nakano Almada, Kauan Divino Pouso Mariano, Maykon Adriell Dutra, Victor Emanuel da Silva Monteiro, Juliana Resplande Sant'Anna Gomes, Arlindo Rodrigues Galvão Filho, Anderson da Silva Soares

机构 * Institute of Informatics, Federal University of Goiás, Brazil（信息学院，戈亚斯联邦大学，巴西） ； Advanced Knowledge Center in Immersive Technology (AKCIT), Federal University of Goiás, Brazil（沉浸式技术高级知识中心（AKCIT），戈亚斯联邦大学，巴西） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；prompting(title,abstract)；large language model(abstract)；language model(abstract)

Comments 15 pages, 2 figures

Llamarine: Open-source Maritime Industry-specific Large Language Model

William Nguyen, An Phan, Konobu Kimura, Hitoshi Maeno, Mika Tanaka, Quynh Le, William Poucher, Christopher Nguyen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Comments Work in progress

Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models

Ilias Stogiannidis, Stavros Vassos, Prodromos Malakasiotis, Ion Androutsopoulos

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Short paper (5 pages), accepted at Findings of EMNLP 2023

Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring

谁来评判评判者？基于指标的治理：面向持续LLM合规监控的运行时框架

Jehanne Dussert

机构 * Independent Researcher（独立研究者） 纠错

专题命中 预训练与数据 ：LLM(title,title_cn)；language model(abstract)；small language model(abstract)；SLM(abstract_cn)

AI总结 针对AI合规作为审计时二元判定而非生产系统持续可测量属性的问题，提出基于指标的治理原则，并开发开源框架govllm，通过运行时可观测性信号实现持续合规监控，验证了多模型陪审团设计在监管评估中的有效性。

Comments 41 pages, 8 figures, preprint

Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity

对比LLM和微调模型在不同提示复杂度下的NVDRS情境提取性能

Geoffrey Martin, Xuan Zhong Feng, Yifan Peng

机构 * Department of Population Health Sciences, Weill Cornell Medicine（人群健康科学系，威尔·康奈尔医学学院） ； Systems Engineering, Cornell University（系统工程，康奈尔大学） 纠错

专题命中 预训练与数据 ：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 本文研究了在不同提示复杂度下，LLM与微调模型在NVDRS情境提取任务中的表现差异，提出了一种复杂度评分算法，并展示了一个混合方法，通过不同情境选择提示策略，发现LLM在低 prevalence 情境中表现更优，且框架能跨不同前沿LLM通用。

Comments Accepted at IEEE ICHI 2026

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining

Ping Guo, Yubing Ren, Binbin Liu, Fengze Liu, Haobin Lin, Yifan Zhang, Bingni Zhang, Taifeng Wang, Yin Zheng

机构 * ByteDance（字节跳动） ； Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(title)；分类 cs.CL、cs.AI

Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPs

Jacob Portes, Connor Jennings, Erica Ji Yuen, Sasha Doubov, Michael Carbin

机构 * Databricks Mosaic Research（Databricks Mosaic研究） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(title)；language model(title)；LLM(abstract)

Comments 15 pages, 4 figures

CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models

Guang Liu, Liangdong Wang, Jijie Li, Yang Yu, Yao Xu, Jiabei Chen, Yu Bai, Feng Liao, Yonghua Lin

机构 * Data Research Team（数据研究团队） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(title)；language model(title)；LLM(abstract)

KnowledgePrompts: Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting

Thilini Wijesiriwardene, Ruwan Wickramarachchi, Sreeram Vennam, Vinija Jain, Aman Chadha, Amitava Das, Ponnurangam Kumaraguru, Amit Sheth

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(title)；分类 cs.CL、cs.AI

Comments Accepted at COLING 2025

All Languages Matter: On the Multilingual Safety of Large Language Models

Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, Michael R. Lyu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；prompting(abstract)

Comments Accepted by ACL 2024 Findings. The first multilingual safety benchmark for large language models

LLMR: Real-time Prompting of Interactive Worlds using Large Language Models

Fernanda De La Torre, Cathy Mengying Fang, Han Huang, Andrzej Banburski-Fahey, Judith Amores Fernandez, Jaron Lanier

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(title)；分类 cs.CL、cs.AI

Comments 46 pages, 18 figures; Matching version accepted at CHI 2024

Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models

Fangkai Jiao, Bosheng Ding, Tianze Luo, Zhanfeng Mo

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(title)；分类 cs.CL、cs.AI

AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguese

AMALIA技术报告：一个完全开源的大型语言模型用于葡萄牙语（欧洲葡萄牙语）

Afonso Simplício, Gonçalo Vinagre, Miguel Moura Ramos, Diogo Tavares, Rafael Ferreira, Giuseppe Attanasio, Duarte M. Alves, Inês Calvo, Inês Vieira, Rui Guerra, James Furtado, Beatriz Canaverde, Iago Paulo, Vasco Ramos, Diogo Glória-Silva, Miguel Faria, Marcos Treviso, Daniel Gomes, Pedro Gomes, David Semedo, André Martins, João Magalhães

机构 * NOVA School of Science and Technology（新大学科学与技术学院） ； NOVA LINCS（新大学LINCS实验室） ； Instituto de Telecomunicações（电信研究所） ； Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院） ； Fundação para a Ciência e Tecnologia（科学技术基金会） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

AI总结 AMALIA是一个完全开源的大型语言模型，专注于欧洲葡萄牙语，通过增加高质量数据提升模型性能，并发布新基准测试以评估葡萄牙语生成和语言能力。

Comments PROPOR 2026 - The 17th International Conference on Computational Processing of Portuguese

Large Language Models for Travel Behavior Prediction

基于大语言模型的出行行为预测

Baichuan Mo, Hanyong Xu, Ruoyun Ma, Jung-Hoon Cho, Dingyi Zhuang, Xiaotong Guo, Jinhua Zhao

机构 * Department of Civil Engineering, Tsinghua University, Beijing, China（清华大学土木工程系） ； Department of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139（麻省理工学院土木与环境工程系） ； Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, MA 20139（麻省理工学院城市研究与规划系） ； Department of Management Science and Engineering, Stanford University, Stanford, CA 94305（斯坦福大学管理科学与工程系） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文提出利用大语言模型进行出行行为预测，通过零样本提示和文本嵌入两种方法，实现了与传统模型相当的预测性能。

Can Safety Emerge from Weak Supervision? A Systematic Analysis of Small Language Models

安全能否从弱监督中涌现？小型语言模型的系统分析

Punyajoy Saha, Sudipta Halder, Debjyoti Mondal, Subhadarshi Panda

机构 * Samsung Research Institute（三星研究院） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；small language model(title,abstract)；large language model(abstract)；preference optimization(abstract)

AI总结 Self-MOA通过弱监督实现小型语言模型的安全对齐，显著提升安全性的同时保持有用性，减少对人工标注数据的依赖。

Comments 19 pages, 10 tables, 7 figures, under Review

STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions

STELLA: 通过语义抽象引导大型语言模型进行时间序列预测

Junjie Fan, Hongye Zhao, Linduo Wei, Jiayu Rao, Guijia Li, Jiaxin Yuan, Wenqi Xu, Yong Qi

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 STELLA通过动态语义抽象机制，提升大型语言模型在时间序列预测中的表现，实现更精准的长期和短期预测。

Comments This work has been submitted to the IEEE for possible publication

Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimization

Wataru Kawakami, Keita Suzuki, Junichiro Iwasawa

机构 * Preferred Networks Inc.（Preferred Networks公司） ； Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Fourier Head: Helping Large Language Models Learn Complex Probability Distributions

Nate Gillman, Daksh Aggarwal, Michael Freeman, Saurabh Singh, Chen Sun

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Comments Camera ready version (ICLR 2025). Code at https://nategillman.com/fourier-head

LoLCATs: On Low-Rank Linearizing of Large Language Models

Michael Zhang, Simran Arora, Rahul Chalamala, Alan Wu, Benjamin Spector, Aaryan Singhal, Krithik Ramesh, Christopher Ré

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments 58 pages, 25 figures, 26 tables, ICLR 2025

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data

Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, William Yang Wang

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Accepted to ICLR 2025

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective

Zeyu Gan, Yong Liu

专题命中 预训练与数据 ：LLM(title,abstract)；post-training(title,abstract)；large language model(abstract)；language model(abstract)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining

Ce Ge, Zhijian Ma, Daoyuan Chen, Yaliang Li, Bolin Ding

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Clarify details

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability

Zhuoyan Xu, Zhenmei Shi, Yingyu Liang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models

Yixin Ou, Ningyu Zhang, Honghao Gui, Ziwen Xu, Shuofei Qiao, Yida Xue, Runnan Fang, Kangwei Liu, Lei Li, Zhen Bi, Guozhou Zheng, Huajun Chen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；prompting(abstract)

Comments ACL 2024 System Demonstrations; Project website: https://zjunlp.github.io/project/EasyInstruct Code: https://github.com/zjunlp/EasyInstruct Video: https://youtu.be/rfQOWYfziFo Demo: https://huggingface.co/spaces/zjunlp/EasyInstruct

In-context Pretraining: Language Modeling Beyond Document Boundaries

Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li, Gergely Szilvasy, Rich James, Xi Victoria Lin, Noah A. Smith, Luke Zettlemoyer, Scott Yih, Mike Lewis

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；prompting(abstract)

GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models

Ruotong Liao, Xu Jia, Yangzhe Li, Yunpu Ma, Volker Tresp

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；instruction tuning(abstract)

Comments 14 pages, Findings of NAACL 2024, Spotlight on TGL@NeurIPS2023

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Dahyun Kim, Chanjun Park, Sanghoon Kim, Wonsung Lee, Wonho Song, Yunsu Kim, Hyeonwoo Kim, Yungi Kim, Hyeonju Lee, Jihoo Kim, Changbae Ahn, Seonghoon Yang, Sukyung Lee, Hyunbyung Park, Gyoungjin Gim, Mikyoung Cha, Hwalsuk Lee, Sunghun Kim

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments accepted to NAACL 2024 Industry Track

SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models

Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, Jiaming Han, Siyuan Huang, Yichi Zhang, Xuming He, Hongsheng Li, Yu Qiao

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Comments Work in progress. Code and demos are released at https://github.com/Alpha-VLLM/LLaMA2-Accessory

Planting a SEED of Vision in Large Language Model

Yuying Ge, Yixiao Ge, Ziyun Zeng, Xintao Wang, Ying Shan

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；instruction tuning(abstract)

Comments Technical Report; Project released at: https://github.com/AILab-CVC/SEED

Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection

Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, Hongxia Jin

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；instruction tuning(abstract)

Comments Accepted to NAACL 2024. Project page: https://poison-llm.github.io