语言大模型 / LLM

Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, Junyang Lin

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Code is available at https://github.com/QwenLM/Qwen2-VL. arXiv admin note: text overlap with arXiv:2408.15262 by other authors

Can Active Label Correction Improve LLM-based Modular AI Systems?

Karan Taneja, Ashok Goel

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.AI、cs.LG

Comments EMNLP (Main) 2024, 13 pages, 6 figures

MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages

Marco Gaido, Sara Papi, Luisa Bentivogli, Alessio Brutti, Mauro Cettolo, Roberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2024 Main Conference

Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models

Laura Bravo-Sánchez, Jaewoo Heo, Zhenzhen Weng, Kuan-Chieh Wang, Serena Yeung-Levy

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Project webpage: https://laubravo.github.io/apu_website/

FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model

Zezheng Song, Jiaxin Yuan, Haizhao Yang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models

Eitan Wagner, Yuli Slavutsky, Omri Abend

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering

Chang Zong, Yuchen Yan, Weiming Lu, Jian Shao, Eliot Huang, Heng Chang, Yueting Zhuang

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, Accepted by EMNLP 2024

Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization

Mehrdad Zakershahrak, Samira Ghodratnama

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Unlocking Potential Binders: Multimodal Pretraining DEL-Fusion for Denoising DNA-Encoded Libraries

Chunbin Gu, Mutian He, Hanqun Cao, Guangyong Chen, Chang-yu Hsieh, Pheng Ann Heng

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang, Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, Xiren Zhou

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 24 pages

MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs

Xingtong Yu, Chang Zhou, Yuan Fang, Xinming Zhang

专题命中 预训练与数据 ：prompting(title,abstract)；分类 cs.CL、cs.LG

Comments WWW2024 research track

AnyGraph: Graph Foundation Model in the Wild

Lianghao Xia, Chao Huang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis

Seunghwan An, Gyeongdong Woo, Jaesung Lim, ChangHyun Kim, Sungchul Hong, Jong-June Jeon

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

REFINE-LM: Mitigating Language Model Stereotypes via Reinforcement Learning

Rameez Qureshi, Naïm Es-Sebbani, Luis Galárraga, Yvette Graham, Miguel Couceiro, Zied Bouraoui

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

LiPCoT: Linear Predictive Coding based Tokenizer for Self-supervised Learning of Time Series Data via Language Models

Md Fahim Anjum

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 5 figures

MooER: LLM-based Speech Recognition and Translation Models from Moore Threads

Junhao Xu, Zhenlin Liang, Yi Liu, Yichao Hu, Jian Li, Yajun Zheng, Meng Cai, Hua Wang

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Language Modeling with Reduced Densities

Tai-Danae Bradley, Yiannis Vlassopoulos

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 21 pages; v2: added reference; v3: revised abstract and introduction for clarity; v4: Compositionality version

Journal ref Compositionality, Volume 3 (2021) (November 30, 2021) compositionality:13514

Do language models plan ahead for future tokens?

Wilson Wu, John X. Morris, Lionel Levine

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 24 pages, 11 figures. Camera-ready for COLM 2024

A New Type of Foundation Model Based on Recordings of People's Emotions and Physiology

David Gamez, Dionis Barcari, Aliya Grig

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 2 figures, 3 tables

Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERT

Muhammad Ali, Swetasudha Panda, Qinlan Shen, Michael Wick, Ari Kobren

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.AI

Embedding And Clustering Your Data Can Improve Contrastive Pretraining

Luke Merrick

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments 16 pages, 3 figures, 2 tables

MMPolymer: A Multimodal Multitask Pretraining Framework for Polymer Property Prediction

Fanmeng Wang, Wentao Guo, Minjie Cheng, Shen Yuan, Hongteng Xu, Zhifeng Gao

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by the 33rd ACM International Conference on Information and Knowledge Management (CIKM 2024)

Region-centric Image-Language Pretraining for Open-Vocabulary Detection

Dahun Kim, Anelia Angelova, Weicheng Kuo

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments ECCV 2024, project page at https://github.com/google-research/google-research/tree/master/fvlm/dito

Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition

Gagan Bhatia, El Moatez Billah Nagoudi, Fakhraddin Alwajih, Muhammad Abdul-Mageed

专题命中 预训练与数据 ：LLM(title)；foundation model(abstract)；分类 cs.CL、cs.AI

On Pretraining Data Diversity for Self-Supervised Learning

Hasan Abed Al Kader Hammoud, Tuhin Das, Fabio Pizzati, Philip Torr, Adel Bibi, Bernard Ghanem

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments ECCV 2024

QuRating: Selecting High-Quality Data for Training Language Models

Alexander Wettig, Aatmik Gupta, Saumya Malik, Danqi Chen

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at ICML 2024. The results for top-k selection have been corrected. The code, models and data are available at https://github.com/princeton-nlp/QuRating

NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretraining

Chenguo Lin, Xumeng Wen, Wei Cao, Congrui Huang, Jiang Bian, Stephen Lin, Zhirong Wu

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by TMLR 2024

Synthetic Data Aided Federated Learning Using Foundation Models

Fatima Abacha, Sin G. Teo, Lucas C. Cordeiro, Mustafa A. Mustafa

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Uni-Mol2: Exploring Molecular Pretraining Model at Scale

Xiaohong Ji, Zhen Wang, Zhifeng Gao, Hang Zheng, Linfeng Zhang, Guolin Ke, Weinan E

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Zero-shot prompt-based classification: topic labeling in times of foundation models in German Tweets

Simon Münker, Kai Kugler, Achim Rettinger

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 2 tables, 1 figure