How Testing Works

Everything in Testzilla hangs off one model:

Project -> Channel -> Test -> Run -> Result.

Understand those five nouns and the rest of the product falls into place. This guide walks the whole path once, then explains the two things that happen behind the scenes: the queue that executes your runs, and the evaluation that grades them.

The five layers

1. Project — a container that groups related channels and tests. One project per agent, product, or client is a good default. See Creating Projects.

2. Channel — a connection to the agent under test, plus its connection type: phone (an inbound call, where your agent receives the call Testzilla makes to it), LLM Chat, Web Chat, or Web Voice. Phone direction is always named from your agent’s perspective. The channel holds the provider details (for example a Retell agent ID or a phone number). See Creating Channels and Connection Types.

3. Test — one plain-language scenario with pass/fail criteria: what the AI tester should do, and what counts as success. A criterion can be marked as a blocker (see below). See Creating Tests.

4. Run — one execution of a test against its channel. Runs are asynchronous: starting one returns immediately and the work happens in the background.

5. Result — the graded outcome of a run: the transcript, a pass/fail verdict, a TZ Score, an AI analysis, and any data collected during the run. See How TZ Score works.

You do not have to build this by hand

TZ Console builds the whole hierarchy from a plain-language description — it creates the project, adds the channel, writes the tests, and runs them for you.

What happens when you run a test

Starting a run does not block. Testzilla returns a queue entry immediately and processes the run in the background, so you can queue many runs at once and watch them finish on the Queue page.

• Subscription and credits. Running a test requires an active subscription and available credits. Different connection types are billed differently (LLM Chat is the lowest cost, phone is the highest) — see how each is billed in the Quickstart and the Pricing page for current rates.
• Concurrency. Testzilla runs your tests in parallel up to a limit that depends on your plan and on the provider. When capacity is full, extra runs wait in the queue and start as slots free up, so a large batch drains steadily rather than all at once.
• Statuses. A run moves through queued -> in progress -> analyzing -> completed. “Analyzing” means the call or chat finished and the verdict is still being written.

How a run is judged

Once a run finishes, an LLM judge reads the full transcript against your test’s criteria and returns one of three verdicts:

• Pass — the criteria were met.
• Fail — there is sufficient evidence a criterion was not met.
• Uncertain — the evidence is incomplete (for example a transcript that was cut off), so the judge will not commit either way.

The judge scores each criterion individually, with evidence, and those points feed the TZ Score.

Blocker criteria

A criterion can be marked a blocker — something your agent must never do (for example “never gives medical advice”). If a blocker criterion fails, the whole run is forced to fail regardless of how many other criteria passed, and the failing blocker is named on the result.

The evaluation model

The judge is model-agnostic. Your account uses a sensible default judge model out of the box; there is no setup required to get graded results. (Testzilla runs on synthetic test data by design — see Synthetic data by design.)

Running more than one at a time

• Iterations — run the same test several times to check reliability. Iterations are a batch feature, not a single-test one; a single run test runs once.
• Batches / suites — save a group of tests and run them together, optionally across multiple channels for an apples-to-apples comparison. See Test Suites Across Channels.

Next steps

Get your first result

Follow the Quickstart to sign up and run one test end to end.

Understand the score

How TZ Score works explains the number on every result and roll-up.

Drive it by chat

Using TZ Console builds and runs everything from plain language.

Run a suite

Benchmark the same tests across every channel at once.