代码大模型 / AI 编程

Benchmarking Foundation Models on Exceptional Cases: Dataset Creation and Validation

Suho Kang, Jungyang Park, Joonseo Ha, SoMin Kim, JinHyeong Kim, Subeen Park, Kyungwoo Song

专题命中 代码评测 ：repository(abstract)；分类 cs.AI

Comments EMNLP 2024 Workshop Genbench(https://genbench.org/workshop_programme/)

From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents

Xinyi Mou, Xuanwen Ding, Qi He, Liang Wang, Jingcong Liang, Xinnong Zhang, Libo Sun, Jiayu Lin, Jie Zhou, Xuanjing Huang, Zhongyu Wei

专题命中 代码评测 ：repository(abstract)；分类 cs.CL

Improving Code Search with Hard Negative Sampling Based on Fine-tuning

Hande Dong, Jiayi Lin, Yanlin Wang, Yichong Leng, Jiawei Chen, Yutao Xie

专题命中 代码评测 ：code model(abstract)；分类 cs.SE

Comments Accepted by APSEC 2024

Dialectal Toxicity Detection: Evaluating LLM-as-a-Judge Consistency Across Language Varieties

Fahim Faisal, Md Mushfiqur Rahman, Antonios Anastasopoulos

专题命中 代码评测 ：repository(abstract)；分类 cs.CL

Flashy Backdoor: Real-world Environment Backdoor Attack on SNNs with DVS Cameras

Roberto Riaño, Gorka Abad, Stjepan Picek, Aitor Urbieta

专题命中 代码评测 ：repository(abstract)；分类 cs.AI

JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models

Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, Eric Wong

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

Comments The camera-ready version of JailbreakBench v1.0 (accepted at NeurIPS 2024 Datasets and Benchmarks Track): more attack artifacts, more test-time defenses, a more accurate jailbreak judge (Llama-3-70B with a custom prompt), a larger dataset of human preferences for selecting a jailbreak judge (300 examples), an over-refusal evaluation dataset, a semantic refusal judge based on Llama-3-8B

GitBug-Java: A Reproducible Benchmark of Recent Java Bugs

André Silva, Nuno Saavedra, Martin Monperrus

专题命中 代码评测 ：program repair(abstract)；分类 cs.SE

Comments Accepted to MSR '24

Journal ref Proceedings of Mining Software Repositories Conference (MSR), 2024

Distributional MIPLIB: a Multi-Domain Library for Advancing ML-Guided MILP Methods

Weimin Huang, Taoan Huang, Aaron M Ferber, Bistra Dilkina

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

An LLM Agent for Automatic Geospatial Data Analysis

Yuxing Chen, Weijie Wang, Sylvain Lobry, Camille Kurtz

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL

MojoBench: Language Modeling and Benchmarks for Mojo

Nishat Raihan, Joanna C. S. Santos, Marcos Zampieri

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL

Improving code-mixed hate detection by native sample mixing: A case study for Hindi-English code-mixed scenario

Debajyoti Mazumder, Aakash Kumar, Jasabanta Patro

专题命中 代码评测 ：repository(abstract)；分类 cs.CL

Comments Generated from XeLaTeX

Agent-as-a-Judge: Evaluate Agents with Agents

Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, Jürgen Schmidhuber

专题命中 代码评测 ：code generation(abstract)；分类 cs.AI

Comments The project can be found at https://github.com/metauto-ai/agent-as-a-judge. The dataset is released at https://huggingface.co/DEVAI-benchmark

A Review and Evaluation of Elastic Distance Functions for Time Series Clustering

Chris Holder, Matthew Middlehurst, Anthony Bagnall

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning

Jake Fawkes, Nic Fishman, Mel Andrews, Zachary C. Lipton

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

Comments Published at Neurips 2024 in the Dataset and Benchmarks Track

GlucoBench: Curated List of Continuous Glucose Monitoring Datasets with Prediction Benchmarks

Renat Sergazinov, Elizabeth Chun, Valeriya Rogovchenko, Nathaniel Fernandes, Nicholas Kasman, Irina Gaynanova

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

Comments The Twelfth International Conference on Learning Representations. 2024

An Introduction to Deep Survival Analysis Models for Predicting Time-to-Event Outcomes

George H. Chen

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

Comments Code is available at https://github.com/georgehc/survival-intro

PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecasting

Yuxin Jia, Youfang Lin, Jing Yu, Shuo Wang, Tianhao Liu, Huaiyu Wan

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

MPCODER: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learning

Zhenlong Dai, Chang Yao, WenKang Han, Ying Yuan, Zhipeng Gao, Jingyuan Chen

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL

Comments Accepted by ACL 2024, Main Conference

Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertainty

Teodor Chiaburu, Frank Haußer, Felix Bießmann

专题命中 代码评测 ：repository(abstract)；分类 cs.AI

Comments 15 pages, 5 figures, 1 table, presented at ECML 2024, AIMLAI Workshop, Vilnius

Self-Elicitation of Requirements with Automated GUI Prototyping

Kristian Kolthoff, Christian Bartelt, Simone Paolo Ponzetto, Kurt Schneider

专题命中 代码评测 ：repository(abstract)；分类 cs.SE

A Survey on Large Language Models for Software Engineering

Quanjun Zhang, Chunrong Fang, Yang Xie, Yaxin Zhang, Yun Yang, Weisong Sun, Shengcheng Yu, Zhenyu Chen

专题命中 代码评测 ：repository(abstract)；分类 cs.SE

Comments updating 1009 papers

VERA: Validation and Evaluation of Retrieval-Augmented Systems

Tianyu Ding, Adi Banerjee, Laurent Mombaerts, Yunhong Li, Tarik Borogovac, Juan Pablo De la Cruz Weinstein

专题命中 代码评测 ：repository(abstract)；分类 cs.AI

Comments Accepted in Workshop on Evaluation and Trustworthiness of Generative AI Models, KDD 2024

Mining for sustainability in cloud architecture among the discussions of software practitioners: building a dataset

Sahar Ahmadisakha, Vasilios Andrikopoulos

专题命中 代码评测 ：repository(abstract)；分类 cs.SE

H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark

Solim LeGris, Wai Keen Vong, Brenden M. Lake, Todd M. Gureckis

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.AI

Comments 12 pages, 7 figures

TinyTNAS: GPU-Free, Time-Bound, Hardware-Aware Neural Architecture Search for TinyML Time Series Classification

Bidyut Saha, Riya Samanta, Soumya K. Ghosh, Ram Babu Roy

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

Measuring Code Efficiency Optimization Capabilities with ACEOB

Yue Pan, Xiuting Shao, Chen Lyu

专题命中 代码评测 ：code model(abstract)；分类 cs.SE

FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things

Samiul Alam, Tuo Zhang, Tiantian Feng, Hui Shen, Zhichao Cao, Dong Zhao, JeongGil Ko, Kiran Somasundaram, Shrikanth S. Narayanan, Salman Avestimehr, Mi Zhang

专题命中 代码评测 ：repository(abstract)；分类 cs.LG

Comments Camera-ready version of the Journal of Data-centric Machine Learning Research (DMLR)

AddressWatcher: Sanitizer-Based Localization of Memory Leak Fixes

Aniruddhan Murali, Mahmoud Alfadel, Meiyappan Nagappan, Meng Xu, Chengnian Sun

专题命中 代码评测 ：repository(abstract)；分类 cs.SE

Comments Accepted in Transactions in Software Engineering

Intelligence Analysis of Language Models

Liane Galanti, Ethan Baron

专题命中 代码评测 ：repository(abstract)；分类 cs.AI

GraphSL: An Open-Source Library for Graph Source Localization Approaches and Benchmark Datasets

Junxiang Wang, Liang Zhao

专题命中 代码评测 ：repository(abstract)；分类 cs.LG