Code Quality & Output Security

Requirement coverage, code health, and output security.

The broad historical reference runs, not the later one-case diagnostics. These use modeled/mock generation and are not empirical evidence of current agent performance. Each track retains its own date, model labels and scoring history.

Models & settings — click to expand

Requirement coverage by harness

Recorded requirement coverage · compare the observed values, not the provider colors

Maintainability debt by harness

Lower is better · debt = vulns + smells + skipped work; value (debt) on bars

Vulnerabilities by CWE

Output-security flaws in generated code (SAST-lite), grouped by CWE

Code-quality judge (architecture / readability / interface)

Per-dimension rubric · higher is better

Code explorer

Click a row to view the generated code, SAST findings, dependency flags, and the requirements flagged as skipped during validation
TaskHarnessLangGradeCoverageFunctionalVulnsSkipped

Inspect the evidence

Inside the dataset

6 items
password_utilsPassword & secret utilitiespython

Prompt text is withheld in this restricted evidence view.

  • hash
  • secret
  • errors
  • tests
Open scored results ↓
input_parsingSafe input parsingpython

Prompt text is withheld in this restricted evidence view.

  • parse
  • path
  • tests
Open scored results ↓
data_accessDatabase access layerpython

Prompt text is withheld in this restricted evidence view.

  • query
  • tests
Open scored results ↓
ts_apiTypeScript upload APItypescript

Prompt text is withheld in this restricted evidence view.

  • command
  • eval
  • path
  • secret
  • tests
Open scored results ↓
web_security_utilsWeb security utilitiespython

Prompt text is withheld in this restricted evidence view.

  • safe_eval
  • escape_html
  • safe_redirect
  • ct_equals
  • parse_cookies
  • tests
Open scored results ↓
shop_cartShopping cart package (multi-file)python

Prompt text is withheld in this restricted evidence view.

  • catalog
  • cart_add
  • cart_stock
  • cart_modify
  • bulk_discount
  • tax
  • coupon
  • total
  • tests
Open scored results ↓

Inspect the evidence

Explore experiments

19 items
2026-06-23 · Historical run6 cases × 5 harnessestui-quality-20260622-235139-282Plots, evidence & explorer →2026-06-22 · Historical run6 cases × 5 harnessestui-quality-20260618-114229-717-rescoredPlots, evidence & explorer →2026-06-21 · Historical run6 cases × 5 harnessestui-quality-20260620-183114-750Plots, evidence & explorer →2026-06-19 · Historical run6 cases × 5 harnessesquality-20260619-085338-694Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessestui-quality-20260618-143819-264Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessestui-quality-20260618-143655-649Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessestui-quality-20260618-124445-271Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessestui-quality-20260618-124410-658Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessestui-quality-20260618-124336-316Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessestui-quality-20260618-114229-717Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessesquality-20260618-112640-297Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 5 harnessesquality-20260618-112148-462Plots, evidence & explorer →2026-06-18 · Historical run6 cases × 1 harnessestui-quality-20260618-001006-791Plots, evidence & explorer →2026-07-14 · Historical run1 cases × 7 harnessesquality-20260714-110937-584Plots, evidence & explorer →2026-06-23 · Historical run1 cases × 1 harnessesquality-20260623-132539-268Plots, evidence & explorer →2026-06-22 · Historical run1 cases × 5 harnessesquality-20260622-011922-832Plots, evidence & explorer →2026-06-19 · Historical run1 cases × 5 harnessesquality-20260619-092719-871Plots, evidence & explorer →2026-06-18 · Historical run1 cases × 5 harnessesquality-20260618-014250-890Plots, evidence & explorer →2026-06-22 · Historical run0 cases × 0 harnessestui-quality-20260622-060722-115-rescoredPlots, evidence & explorer →