Promptfoo

You start with a configuration file that names your providers, your prompts and your test cases. Assertions can be deterministic (contains, equals, a JSON schema match, a regex), or model-graded, where another model judges whether an answer satisfies a rubric you wrote. Running the evaluation produces a matrix of prompt against model against test, so you can see which combination actually regressed rather than guessing from an aggregate score.

Red teaming runs from the same configuration but inverts the goal. You point it at your application, choose which attack plugins to enable, and it generates adversarial probes rather than your own test cases, then reports a vulnerability list with severity and a remediation suggestion per finding. The free tier allows ten thousand probes a month.

Both modes are designed to run in CI, so the gate on merging a prompt change is the same shape as the gate on merging code.

• •Declarative evaluations - Test cases and assertions live in YAML next to the code, version controlled and reviewable, rather than in a notebook or a vendor dashboard.
• •Model-graded assertions - Where a deterministic check cannot express the requirement, another model grades the answer against a rubric you write, which covers the large class of outputs that are correct in several wordings.
• •Automated red teaming - Generates jailbreak, prompt-injection, PII-extraction and tool-misuse probes against your own app and produces a vulnerability report with severity and remediation notes.
• •Provider-agnostic - Runs against every major model provider and your own endpoints, so comparing two vendors is a configuration change rather than a rewrite.
• •Local and self-hosted by default - The Community build runs entirely on your machine or your infrastructure, with no requirement to ship prompts or outputs to a third party.
• •CI integration - Both evaluation and red teaming run headless with a non-zero exit on failure, so a prompt change can be gated the same way a code change is.

Teams shipping an LLM feature that real users touch, and particularly anyone who has already been burned by a model upgrade silently changing behaviour. It suits developers who want tests in the repository rather than results in a vendor dashboard, security engineers who need evidence about an agent before it goes live, and anyone comparing two providers who wants the comparison to be reproducible.

It is less useful if you are prototyping and have no stable expectations to assert yet, since writing test cases for a prompt you are still rewriting hourly is wasted effort. Be aware of where the commercial line sits: continuous monitoring, the centralised security and compliance dashboard, team sharing, SSO and the guardrails product are all Enterprise, both paid tiers are Custom with no published figure, and the only route to one is a sales call. The free red team allowance of ten thousand probes a month is real but modest once you are scanning more than a single application. The social proof on the site is anonymised by design, which makes it unverifiable.