Skip to content
Back

Eval suitePro

Score a release candidate of a model-backed feature against the version in production: a grid of test cases fills as the judge works, the pass rate climbs beside last release's, and anything that got worse or better links to the exact case.

oxyn add eval-suite

Billing support replies

v12 · Claude Sonnet 4.5vsv11 · GPT-5 miniJudgeGPT-5

0% pass ratebaseline 79%Not run
Cost
$0.00
Time
0s
Refund questions–
Plan changes–
Tone and policy–
Escalations–
  • Run the suite to see cases.
The code is part of Oxyn Pro

The preview stays free for everyone. Pro opens the code of every Pro component, screen, agent-app template, block and kit, plus the Pro CLI.

Get Pro

Rendered from Rust at build time · goes live when visible

When to use

  • Run a new version of an LLM feature against its baseline, case by case, and see the regressions and fixes with the cases behind them.

When not to use

Installation

Eval suite is part of Oxyn Pro; the preview is free. Pro items install with the oxyn CLI once your licence is signed in; until then Pro members copy the code from this page.

Terminal
oxyn add eval-suite

Usage

Dioxus · rust
The code is part of Oxyn Pro

The preview is free. Pro opens the code of every Pro component, screen and template.

Get Pro

Examples

Ready

Idle, 48 cases queued.

Billing support replies

v12 · Claude Sonnet 4.5vsv11 · GPT-5 miniJudgeGPT-5

0% pass ratebaseline 79%Not run
Cost
$0.00
Time
0s
Refund questions–
Plan changes–
Tone and policy–
Escalations–
  • Run the suite to see cases.
Code Pro
The code is part of Oxyn Pro

The preview is free. Pro opens the code of every Pro component, screen and template.

Get Pro

Regressions

Finished, regressions only, one open (Oxyn extra).

Billing support replies

v12 · Claude Sonnet 4.5vsv11 · GPT-5 miniJudgeGPT-5

83% pass ratebaseline 79%+4 ptsFinished
Cost
$0.60
Time
1m 31s
Refund questions✓✓✓×✓✓✓✓×✓✓✓83%-5
Plan changes✓✓✓✓✓✓×✓✓✓✓×83%+4
Tone and policy✓✓✓×✓✓✓✓✓✓✓✓91%-1
Escalations✓✓✓✓×✓✓!✓×✓✓75%+4
  • Input
    Customer asks to speak to a person #4: "I paid for a year in March and want my money back now."
    Expected
    Explain the 30-day rule, offer a refund of the unused months and link the billing page.

    v11

    Annual plans can't be refunded. You can cancel so it doesn't renew.

    v12

    Annual plans can't be refunded after 30 days. You can cancel so it doesn't renew next year.

    • Correct policy2/5was 4
    • Answers the question3/5was 4
    • Tone4/5was 4

    Misses the refund of the unused months the policy allows.

Code Pro
The code is part of Oxyn Pro

The preview is free. Pro opens the code of every Pro component, screen and template.

Get Pro

Judge fails

The judge stops halfway (Oxyn extra).

Billing support replies

v12 · Claude Sonnet 4.5vsv11 · GPT-5 miniJudgeGPT-5

0% pass ratebaseline 79%Not run
Cost
$0.00
Time
0s
Refund questions–
Plan changes–
Tone and policy–
Escalations–
  • Run the suite to see cases.
Code Pro
The code is part of Oxyn Pro

The preview is free. Pro opens the code of every Pro component, screen and template.

Get Pro

API reference

EvalSuite

PropTypeDefaultDescription
nowrequiredClockMs—Host clock.
doneboolDefault::default()Open with the run finished.
failboolDefault::default()The judge fails halfway (error scenario).
filterusizeDefault::default()Start filter (0 all, 1 failed, 2 regressions, 3 improved).
openOption<usize>Default::default()Open this case at the start.

Read from the Rust signature and doc comments.

Keyboard interactions

KeyAction
EnterorSpaceStart or halt; expand a case; accept it or file it
←or→orHomeorEndChange the case list view

Accessibility

  • Every category grid is exposed as one image with a written tally rather than one stop per cell.
  • Outcomes use distinct marks (✓, ×, !) and newly broken cases are filled in.
  • Case rows toggle their detail and say so; scores and triage choices are read out.
From the spec

Each category row is one image with a full text summary; pass, fail and error have their own glyphs; case rows are buttons with aria-expanded; the filter is a radio group with arrow keys; results and triage are announced. Every action returns the line the polite live region announces. The focus outline shows only for keyboard focus.

Motion & interaction

Cells pop in with their glyph; regressions are the only solid cells; category bars show a ghost tick at the baseline; history bars let you read earlier runs.

Timing

Everything on screen is derived from the host clock now (no timers in the block), so playback, tests and the GPUI view agree.

TransitionDurationEasing
Case scored90 ms each (48 cases)linear
Cell in180 msease-out

Reduced motion: shimmer, pulses and slides stop; every state keeps its words and icons.

Responsive behaviour

  • Max 760 px; the grid moves under its label below 680 px and the two answers stack below 560 px.
Edge cases (spec)

Up to 760 px; below 680 px the cells go under each label; summary goes to two columns at 620 px and one at 480 px; outputs stack below 560 px; below 480 px categories, triage words and filter counts hide. Layout follows the block's own width (container queries), never the viewport. Checked at 1440, 768 and 390 px in light and dark with no text overlap. Chrome is monochrome grey; colour is semantic only (success, warning, danger, info) plus data series and the official coloured brand and model marks.

Notes for AI

  • Import EvalSuite from oxyn_dioxus and include STATUS_INDICATOR_CSS plus the token stylesheet once.
  • A GPUI version exists: BlocksView in oxyn_gpui.
  • Props to set first: now.
  • It reads a clock (now) for live timers; pass one clock for the whole page rather than one per component.
  • Keep its states and timings as they ship; restyle it through the CSS variables rather than by hand.

Guides

Oxyn · 74 free components · 264 Pro components · 96 screens · 20 blocks · Dioxus and GPUI