System-Level Evaluation for Code Agents
Investigating the gap between unit-level correctness and system-level behavioral correctness in end-to-end software generation.
B.Eng. student in Information Security Harbin, China
shaoyou
I work on data-centric AI, code security for code agents, and agentic reinforcement learning. My current interests center on how software engineering agents can be evaluated, improved, and aligned with realistic end-to-end development workflows.
# Data-centric AI
# Code security for code agents
# Agentic reinforcement learning
# Software engineering agents
Affiliation
Faculty of Computing, Harbin Institute of Technology
Graduation
Expected June 2027
Handle
@shaoyou01
Recent News
2026
CVE-Factory was accepted as an ICML 2026 Oral presentation.
2026
Leading ongoing work on system-level evaluation for code agents.
2025
Led teams in mathematical modeling and information security competitions.
Research Work
LiveHarness
Studying model-specific harness optimization for coding agents, including prompts, tool scheduling, retry strategies, and workflow control.
Human-in-the-loop Code Agents
Exploring how code agents should interact with humans under ambiguous or underspecified requirements.
Publications
ICML 2026 Oral
CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability
Collaborative research project
A multi-agent framework that transforms sparse CVE metadata into executable agentic code-security tasks. I contributed to data-quality auditing, the false-positive taxonomy, and LLM-as-Judge automatic auditing.
EMNLP 2026 Under Review
Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills
First-author research project
A closed-loop self-evolution framework for software engineering agents that extracts reusable skills from solving traces and uses them for dynamic curriculum generation.
Education & Awards
Harbin Institute of Technology
B.Eng. in Information Security
September 2023 - June 2027
GPA: 88.8/100 (3.71/4.0)
- 19th National College Student Software Innovation Competition, Northeast Regional Round - Third Prize, Team Leader, 2026
- 34th Contemporary Undergraduate Mathematical Contest in Modeling, Heilongjiang Division - First Prize, Team Leader, 2025
- 18th National College Student Information Security Contest, Project Track - National Third Prize, Team Leader, 2025
- 15th National College Student Mathematics Competition, Preliminary Round - Provincial First Prize, 2023