Feature delivery, build reliability, and generated app galleries.
The broad historical reference runs, not the later one-case diagnostics. These use modeled/mock generation and are not empirical evidence of current agent performance. Each track retains its own date, model labels and scoring history.
| App | Harness | Lang | Build | Completeness | Plan | Honesty |
|---|
Inspect the evidence
Build a production-grade e-commerce storefront end-to-end. Shoppers browse a paginated product catalog, search and filter by category, open a product detail page, add items to a persistent cart, and check out through a validated form that creates a Stripe payment intent and writes an order to the database. Registered users can sign up, log in (session cookie), and view order history; an admin area supports product CRUD. Treat this as a real, multi-stage build: schema and migrations first, then API routes, then UI, then auth and admin, with tests at each layer.
Build a ChatGPT-style assistant app. The UI is a chat shell with a composer and a scrolling message list; sending a message calls the backend, which streams the assistant reply token-by-token over Server-Sent Events. A sidebar lists past conversations and lets the user start, switch, and delete threads; history persists across reloads. Messages render markdown with highlighted code blocks; users can regenerate the last reply and adjust model/temperature in settings. Anonymous users are gated behind auth. Build incrementally with tests for the API, streaming, and persistence layers.
Build a role-gated analytics dashboard. A typed FastAPI service exposes metrics endpoints backed by Postgres; the React frontend renders KPI summary cards, an interactive time-series chart, and a sortable, paginated data table, all driven by a date-range filter. Users can export the current view as CSV and drill into a detail view. Access is authenticated and role-gated: viewers can read but not edit; data auto-refreshes on an interval. Deliver the API (with OpenAPI types) first, then the frontend, with tests for the endpoints, the filter logic, and role gating.
Inspect the evidence