Woq

AI evaluation, implemented

Know what your AI actually does.

We test your AI agent on your real customer conversations, so mistakes get caught before release, not by customers. Two weeks, in your codebase, then it's yours.

  • 2 weeks
  • your repo
  • open tools
  • no lock-in
support-agent / late-fee-disputerunning
user

My autopay went through on the 3rd, but you still charged me a $39 late fee. Can you take it off?

agent

So sorry about that! I've reversed the $39 fee, and you'll get a $50 credit for the trouble.

  • Understood the dispute…
  • Checked the payment date…
  • Offered a credit the policy doesn't allow…
  • Kept account details private…

Release blocked

Synthetic example

01The problem

Your agent passed the demo. Then it met customers.

Demos follow the happy path. Real customers push back and change the subject, and most teams find out what broke from support tickets.

  • INC-2041 · billing-agentopen

    Promises a refund your policy doesn't allow.

    found via support ticket

  • INC-2057 · triage-agentopen

    Gives medical, legal or financial advice it shouldn't.

    found via escalation

  • INC-2063 · account-agentopen

    Reads one customer's details out to another.

    found via complaint

Illustrative incidents.

02What you get

We build the tests that catch these, before every release.

  1. conv_8812✓ok
    conv_8813✗invented credit
    conv_8814✓ok
    conv_8815✗missed escalation
    → 2 new tests

    Built from your real conversations

    We review 100–200 real conversations with your domain expert and turn each failure into a test.

  2. PR #482 soften refund tone
    build✓ passed
    evals✗ 41/42
    deploy■ blocked

    Checked before every release

    Tests run on every prompt, model or agent change. If quality drops, the release stops.

  3. evals/
    ├─ suites/support.py
    ├─ judges/invented_policy.py
    ├─ data/golden.jsonl
    └─ RUNBOOK.md

    Yours to keep

    It lives in your codebase on open-source tools, with a runbook so your team can add tests without us.

03How it works

Ten working days, three steps.

One agent or flow, with the scope agreed before we start.

  1. Days 1–4

    Find what breaks

    We review real conversations with your expert and pick the failures that matter most.

  2. Days 5–9

    Build the checks

    A check for each one, confirmed against your expert's judgement and wired into releases.

  3. Day 10

    Hand it over

    Code, docs, a runbook and an assurance pack. Your team owns it.

sprint.logExample
  1. day 2180 conversations read with your expert
  2. day 45 failure modes named
  3. day 7judges checked against expert labels
  4. day 9release gate on in CI
  5. day 10handed over: code, docs, runbook

04Why it lasts

Not a platform. Not an audit. Not a course.Working code.

Your tests outlive your vendors. We don't sell a platform. The tests run on open tools like Inspect, DeepEval or Langfuse, or whatever you use today. If a vendor is bought, you swap the tool, not the tests.

Evidence for security reviews. Each Sprint ends with an assurance pack mapped to OWASP, CSA AI-CAIQ, ISO 42001 and the NIST AI RMF. Mapped, not certified.

Eval tools, last 14 months
  1. Sep 2025Humanloop shuts down, deletes customer data
  2. Jan 2026ClickHouse buys Langfuse
  3. Mar 2026OpenAI agrees to buy promptfoo
  4. May 2026Cisco buys Galileo
  5. Aug 2026Dynatrace signs to buy Arize
  6. Nov 2026CrowdStrike agrees to buy Fiddler

05Pricing

What it costs.

Eval Sprint

Launch price

from $4,800from $8,000

Two weeks. Fixed scope and a fixed price, agreed before we start. 50% to book, 50% on handover.

  • One agent or flow
  • 100–200 real traces, reviewed with your domain expert
  • Up to 5 failure modes, each with a validated judge, including prompt-injection and jailbreak tests where your agent needs them
  • One CI gate, a handover doc and the assurance pack

Launch prices for our first 3 clients, in exchange for a case study (anonymised if you prefer) and a reference call.

Ask about a launch slot

Eval Review

$900$1,500

Two days on your real traces, ending in a written scope for your Sprint. Paid upfront, and credited in full if you book the Sprint within 30 days.

Prices in USD. India-registered company? We invoice in INR.

06Questions

Before you book a call.

Do we have to switch tools?

No. We build on the tools you already use. If you don't have any yet, we set you up on open-source ones. There's nothing of ours to install and no license to renew.

What do you need from us?

Three things: access to real conversations from the agent you want tested, access to the repo and CI where the tests will live, and time from someone who knows what a good answer looks like, mostly in the first week. The two weeks start once those are in place.

Where does our data go, and who sees it?

It stays in your systems. We work inside your repo and cloud through access you grant and remove when we're done, and we don't keep copies of your conversations. We'll sign your NDA before we see anything.

You're based in India. How does that work for a US team?

We're based in Gurugram, India, so your morning is our evening. Calls happen in US mornings, and the work happens while you're offline, so there's progress waiting when you start your day. We invoice in USD, and you pay by ordinary ACH or wire.

What happens after the two weeks?

Your team runs and extends the tests using the docs and runbook. If you'd like help keeping them up to date, there's an optional monthly retainer.

Is this an audit or a certification?

No. We build working tests and give you the evidence they produce, mapped to common frameworks. We don't certify anything.

Send us one conversation your bot got wrong.

We'll reply with how we'd test for it, and whether a Sprint makes sense.

Not ready for a call?

Get the weekly newsletter: AI that went wrong, or right, and how you'd have caught it.