代码大模型 / AI 编程

ArxEval: Evaluating Retrieval and Generation in Language Models for Scientific Literature

Aarush Sinha, Viraj Virk, Dipshikha Chakraborty, P. S. Sreeja

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study

Alejandro Velasco, Daniel Rodriguez-Cardenas, Luftar Rahman Alif, David N. Palacio, Denys Poshyvanyk

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

DeCon: Detecting Incorrect Assertions via Postconditions Generated by a Large Language Model

Hao Yu, Tianyu Chen, Jiaming Huang, Zongyang Li, Dezhi Ran, Xinyu Wang, Ying Li, Assaf Marron, David Harel, Yuan Xie, Tao Xie

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.PL

LeetDecoding: A PyTorch Library for Exponentially Decaying Causal Linear Attention with CUDA Implementations

Jiaping Wang, Simiao Zhang, Qiao-Chu He, Yifan Chen

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.LG

Comments The source code of LeetDecoding is hosted at https://github.com/Computational-Machine-Intelligence/LeetDecoding

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks

Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 13 figures

PyG-SSL: A Graph Self-Supervised Learning Toolkit

Lecheng Zheng, Baoyu Jing, Zihao Li, Zhichen Zeng, Tianxin Wei, Mengting Ai, Xinrui He, Lihui Liu, Dongqi Fu, Jiaxuan You, Hanghang Tong, Jingrui He

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering

Ruida Hu, Chao Peng, Jingyi Ren, Bo Jiang, Xiangxin Meng, Qinyun Wu, Pengfei Gao, Xinchen Wang, Cuiyun Gao

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

ClustEm4Ano: Clustering Text Embeddings of Nominal Textual Attributes for Microdata Anonymization

Robert Aufschläger, Sebastian Wilhelm, Michael Heigl, Martin Schramm

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 5 figures, accepted for presentation at IDEAS: 2024 28th International Symposium on Database Engineered Applications, Bayonne, France, August 26-29, 2024

Execution-Based Evaluation of Natural Language to Bash and PowerShell for Incident Remediation

Ngoc Phuoc An Vo, Brent Paulovicks, Vadim Sheinin

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL

Extreme AutoML: Analysis of Classification, Regression, and NLP Performance

Edward Ratner, Elliot Farmer, Brandon Warner, Christopher Douglas, Amaury Lendasse

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 7 figures

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs

Zhihan Liu, Shenao Zhang, Yongfei Liu, Boyi Liu, Yingxiang Yang, Zhaoran Wang

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

A Comprehensive Guide to Explainable AI: From Classical Models to LLMs

Weiche Hsieh, Ziqian Bi, Chuanqi Jiang, Junyu Liu, Benji Peng, Sen Zhang, Xuanhe Pan, Jiawei Xu, Jinlang Wang, Keyu Chen, Pohsun Feng, Yizhu Wen, Xinyuan Song, Tianyang Wang, Ming Liu, Junjie Yang, Ming Li, Bowen Jing, Jintao Ren, Junhao Song, Hong-Ming Tseng, Yichao Zhang, Lawrence K. Q. Yan, Qian Niu, Silin Chen, Yunze Wang, Chia Xin Liang

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

ScratchEval: Are GPT-4o Smarter than My Child? Evaluating Large Multimodal Models with Visual Programming Challenges

Rao Fu, Ziyang Luo, Hongzhan Lin, Zhen Ye, Jing Ma

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphs

Zhuofeng Li, Zixing Gou, Xiangnan Zhang, Zhongyuan Liu, Sirui Li, Yuntong Hu, Chen Ling, Zheng Zhang, Liang Zhao

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2024

BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices

Anka Reuel, Amelia Hardy, Chandler Smith, Max Lamparth, Malcolm Hardy, Mykel J. Kochenderfer

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments Accepted as a Spotlight Poster to NeurIPS 2024

Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs

Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, Zhiqiang Shen

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Camera-ready Version. Website at https://mbzuai-llm.github.io/webpage2code/

InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models

Linyi Li, Shijie Geng, Zhenwen Li, Yibo He, Hao Yu, Ziyue Hua, Guanghan Ning, Siwei Wang, Tao Xie, Hongxia Yang

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.LG

Comments 31 pages. Appear at NeurIPS 2024 Datasets and Benchmarks track. Project website: https://infi-coder.github.io/infibench

Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models

Yeming Wen, Swarat Chaudhuri

专题命中 代码评测 ：code generation(abstract)；分类 cs.AI、cs.LG

Tool Learning with Large Language Models: A Survey

Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, Ji-Rong Wen

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments The article has been accepted by Frontiers of Computer Science (FCS), with the DOI: {10.1007/s11704-024-40678-2}

GRAMMAR: Grounded and Modular Methodology for Assessment of Closed-Domain Retrieval-Augmented Language Model

Xinzhe Li, Ming Liu, Shang Gao

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Under Review

Learning Language Structures through Grounding

Freda Shi

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.CL、cs.AI

Comments Ph.D. Thesis

User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

Szymon Bobek, Paloma Korycińska, Monika Krakowska, Maciej Mozolewski, Dorota Rak, Magdalena Zych, Magdalena Wójcik, Grzegorz J. Nalepa

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers

Hussein Mozannar, Valerie Chen, Mohammed Alsobay, Subhro Das, Sebastian Zhao, Dennis Wei, Manish Nagireddy, Prasanna Sattigeri, Ameet Talwalkar, David Sontag

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.AI

Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People

Xidong Wang, Nuo Chen, Junyin Chen, Yidong Wang, Guorui Zhen, Chunxian Zhang, Xiangbo Wu, Yan Hu, Anningzhe Gao, Xiang Wan, Haizhou Li, Benyou Wang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Preprint

Mark My Words: Analyzing and Evaluating Language Model Watermarks

Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, David Wagner

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 18 figures

MOMENT: A Family of Open Time-series Foundation Models

Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, Artur Dubrawski

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICML'24. This is a revision. See changelog in the Appendix

GEMS: Generative Expert Metric System through Iterative Prompt Priming

Ti-Chung Cheng, Carmen Badea, Christian Bird, Thomas Zimmermann, Robert DeLine, Nicole Forsgren, Denae Ford

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

Comments 29 pages, 3 figures

BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data

Xuwu Wang, Qiwen Cui, Yunzhe Tao, Yiran Wang, Ziwei Chai, Xiaotian Han, Boyi Liu, Jianbo Yuan, Jing Su, Guoyin Wang, Tingkai Liu, Liyu Chen, Tianyi Liu, Tao Sun, Yufeng Zhang, Sirui Zheng, Quanzeng You, Yang Yang, Hongxia Yang

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow

Nathanaël Beau, Benoît Crabbé

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL

Comments Accepted to ACL 2024 Findings

A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations

Jinqiang Wang, Huansheng Ning, Yi Peng, Qikai Wei, Daniel Tesfai, Wenwei Mao, Tao Zhu, Runhe Huang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments 25 pages,4 figures