代码大模型 / AI 编程

How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark

Ruizhong Qiu, Weiliang Will Zeng, James Ezick, Christopher Lott, Hanghang Tong

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments ICLR 2025

CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories

Yijia Xiao, Runhui Wang, Luyang Kong, Davor Golac, Wei Wang

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI、cs.LG

Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey

Liang Chen, Zekun Wang, Shuhuai Ren, Lei Li, Haozhe Zhao, Yunshui Li, Zefan Cai, Hongcheng Guo, Lei Zhang, Yizhe Xiong, Yichi Zhang, Ruoyu Wu, Qingxiu Dong, Ge Zhang, Jian Yang, Lingwei Meng, Shujie Hu, Yulong Chen, Junyang Lin, Shuai Bai, Andreas Vlachos, Xu Tan, Minjia Zhang, Wen Xiao, Aaron Yee, Tianyu Liu, Baobao Chang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 69 papes, 18 figures, repo at https://github.com/LMM101/Awesome-Multimodal-Next-Token-Prediction

ChainStream: An LLM-based Framework for Unified Synthetic Sensing

Jiacheng Liu, Yuanchun Li, Liangyan Li, Yi Sun, Hao Wen, Xiangyu Li, Yao Guo, Yunxin Liu

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments 18 pages, 8 figures

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Yutong Wu, Di Huang, Wenxuan Shi, Wei Wang, Lingzhe Gao, Shihao Liu, Ziyuan Nan, Kaizhao Yuan, Rui Zhang, Xishan Zhang, Zidong Du, Qi Guo, Yewen Pu, Dawei Yin, Xing Hu, Yunji Chen

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Accepted for publication at AAAI 2025. Extended version with full appendix, 18 pages

Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases

Zian Su, Xiangzhe Xu, Ziyang Huang, Kaiyuan Zhang, Xiangyu Zhang

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Open-Source Conversational AI with SpeechBrain 1.0

Mirco Ravanelli, Titouan Parcollet, Adel Moumen, Sylvain de Langen, Cem Subakan, Peter Plantinga, Yingzhi Wang, Pooneh Mousavi, Luca Della Libera, Artem Ploujnikov, Francesco Paissan, Davide Borra, Salah Zaiem, Zeyu Zhao, Shucong Zhang, Georgios Karakasidis, Sung-Lin Yeh, Pierre Champion, Aku Rouhe, Rudolf Braun, Florian Mai, Juan Zuluaga-Gomez, Seyed Mahed Mousavi, Andreas Nautsch, Ha Nguyen, Xuechen Liu, Sangeet Sagar, Jarod Duret, Salima Mdhaffar, Gaelle Laperriere, Mickael Rouvier, Renato De Mori, Yannick Esteve

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the Journal of Machine Learning research (JMLR), Machine Learning Open Source Software

Deep Configuration Performance Learning: A Systematic Survey and Taxonomy

Jingzhi Gong, Tao Chen

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted by the TOSEM survey track in October 2024

Learning Code Preference via Synthetic Evolution

Jiawei Liu, Thanh Nguyen, Mingyue Shang, Hantian Ding, Xiaopeng Li, Yu Yu, Varun Kumar, Zijian Wang

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.LG

CELI: Controller-Embedded Language Model Interactions

Jan-Samuel Wagner, Dave DeCaprio, Abishek Chiffon Muthu Raja, Jonathan M. Holman, Lauren K. Brady, Sky C. Cheung, Hosein Barzekar, Eric Yang, Mark Anthony Martinez, David Soong, Sriram Sridhar, Han Si, Brandon W. Higgs, Hisham Hamadeh, Scott Ogden

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments 26 pages, 2 figures

Representation Learning of Structured Data for Medical Foundation Models

Vijay Prakash Dwivedi, Viktor Schlegel, Andy T. Liu, Thanh-Tung Nguyen, Abhinav Ramesh Kashyap, Jeng Wei, Wei-Hsian Yin, Stefan Winkler, Robby T. Tan

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024 Workshop on Unifying Representations in Neural Models (UniReps 2024)

RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance

Haolin Jin, Zechao Sun, Huaming Chen

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Narrow Transformer: StarCoder-Based Java-LM For Desktop

Kamalkumar Rathinasamy, Balaji A J, Ankush Kumar, Gagan Gayari, Harshini K, Rajab Ali Mondal, Sreenivasa Raghavan K S, Swayam Singh, Mohammed Rafee Tarafdar

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Updated Authors list

Agentic Retrieval-Augmented Generation for Time Series Analysis

Chidaksh Ravuru, Sagar Srinivas Sakhinana, Venkataramana Runkana

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Paper was accepted for Undergraduate Consortium at ACM KDD, 2024. Please find the link: https://kdd2024.kdd.org/undergraduate-consortium/

From Feature Importance to Natural Language Explanations Using LLMs with RAG

Sule Tekkesinoglu, Lars Kunze

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation

Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, Yun-Hsuan Sung

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 31 pages, 5 figures, 7 tables

ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networks

Pranshav Gajjar, Vijay K. Shah

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

DiscoveryBench: Towards Data-Driven Discovery with Large Language Models

Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, Peter Clark

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Website: https://github.com/allenai/discoverybench

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency

Baizhou Huang, Shuai Lu, Weizhu Chen, Xiaojun Wan, Nan Duan

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments ACL 2024

Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code

Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, Rui Wang

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Repo: https://github.com/codefuse-ai/Awesome-Code-LLM. 9 figures, 18 tables, and 902 references. Under review

Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step

Li Zhong, Zilong Wang, Jingbo Shang

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Preprint

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation

Weixiang Yan, Haitian Liu, Yunkun Wang, Yunzhe Li, Qian Chen, Wen Wang, Tingyu Lin, Weishan Zhao, Li Zhu, Hari Sundaram, Shuiguang Deng

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Accepted by ACL 2024 main conference

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, Ion Stoica

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.LG

Comments Website - https://livecodebench.github.io/

Kotlin ML Pack: Technical Report

Sergey Titov, Mikhail Evtikhiev, Anton Shapkin, Oleg Smirnov, Sergei Boytsov, Sergei Boytsov, Dariia Karaeva, Maksim Sheptyakov, Mikhail Arkhipov, Timofey Bryksin, Egor Bogomolov

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.PL

Automated Multi-Language to English Machine Translation Using Generative Pre-Trained Transformers

Elijah Pelofske, Vincent Urias, Lorie M. Liebrock

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

LeTI: Learning to Generate from Textual Interactions

Xingyao Wang, Hao Peng, Reyhaneh Jabbarvand, Heng Ji

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments NAACL 2024 Findings

ICE-Score: Instructing Large Language Models to Evaluate Code

Terry Yue Zhuo

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Accepted to Findings of EACL 2024

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model

Peng Di, Jianguo Li, Hang Yu, Wei Jiang, Wenting Cai, Yang Cao, Chaoyu Chen, Dajun Chen, Hongwei Chen, Liang Chen, Gang Fan, Jie Gong, Zi Gong, Wen Hu, Tingting Guo, Zhichao Lei, Ting Li, Zheng Li, Ming Liang, Cong Liao, Bingchang Liu, Jiachen Liu, Zhiwei Liu, Shaojun Lu, Min Shen, Guangpei Wang, Huan Wang, Zhi Wang, Zhaogui Xu, Jiawei Yang, Qing Ye, Gehao Zhang, Yu Zhang, Zelin Zhao, Xunjin Zheng, Hailian Zhou, Lifu Zhu, Xianying Zhu

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted by ICSE-SEIP 2024

CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, Sida I. Wang

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments 71 pages, 29 figures

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data

Yu Yang, Aaditya K. Singh, Mostafa Elhoushi, Anas Mahmoud, Kushal Tirumala, Fabian Gloeckle, Baptiste Rozière, Carole-Jean Wu, Ari S. Morcos, Newsha Ardalani

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 pages, 4 figures, Oral Presentation at 3rd Workshop on Efficient Natural Language and Speech Processing (ENLSP-III), NeurIPS 2023