We design, build, and runyour QA agent system.
The workflow, the agents, the tools, the guardrails, the knowledge corpus, and the process-mining that audits and improves it — engineered and operated end to end. A production-grade agentic QA capability you own — on your SDK or fully custom, in your environment — and the quality gate for the code your AI writes. No AI-platform team to hire.
- ✓guardrail fence · injection 100%→0%
- ✓resume bridge · #4471 recovered · ECONNRESET
- ✓POM heal · login selector re-bound to testid
- ↺sdk watch · 1.0.20 held · peer-dep flagged
- ✓OCPM · run sound · 0 anti-patterns · audit ✓
architect the workflow, agents, tools, guardrails, knowledge corpus & process-mining
implement all ten layers on your SDK, in your repos
run the fleet — trigger, triage, manage the queue
SDK upgrades, drift heal, reliability incident response
OCPM audit → fix → re-measure; evolve coverage
Standing up a toy QA agent is a weekend. One you can trust is a team-quarter.
The hard part isn't the demo. It's everything that makes an agentic QA system production-grade — and everything that keeps it there. We own both.
It has to be engineered, not prompted.
Most teams get the agent loop working and stop. The result is confidently wrong — and nobody can prove otherwise. A trustworthy system needs all of this:
- ▸A real workflow + state machine — not an ad-hoc loop
- ▸A proper agent roster with sound prompts & autonomy bounds
- ▸A tool layer for test-data setup and app interaction
- ▸Guardrails that actually hold — a string-match hook is not a guard
- ▸A domain-knowledge corpus that makes the agents accurate on your app
- ▸Process-mining (OCPM) to know a run was sound, not just that it ran
- ▸A test suite for the QA system itself
SDK churn
A new SDK version breaks it silently. An unannounced release lands. A peer-dependency bug ships in the “latest” tag. Nobody is watching releases.
Environmental kills
Long runs die with ECONNRESET. A corporate firewall RSTs idle connections at ~300s. Your team burns a day proving it isn't their bug. (It isn't.)
Knowledge drift
Page Object Models and selectors rot the moment the UI changes. The toolkit falls out of sync with the app repo. Accuracy quietly degrades.
Process opacity
It produces a report — but is it routing around the UI, looping, skipping evidence? Without process-level visibility you can't tell sound from confidently-wrong.
Security exposure
Secrets leak into model context. DB credentials are over-broad. The harness is one prompt-injection away from obeying a malicious instruction.
An untested test system
The thing that tests your app has zero tests of its own. Every change to it is a gamble nobody wants to take.
Bus factor
The person who built it moved on, got busy, or never wrote it down. The tribal knowledge evaporated and nobody owns its reliability.
Your AI writes code. Who validates it?
If Cursor, Claude Code, or Copilot write a meaningful share of your production code, your QA system has a second mandate: it is the guardrail between AI-generated code and production. AI-written code fails differently — plausible logic errors, skipped edge cases, subtle behavioral drift — and it ships faster than human review scales.
Govern the testing agents' own conduct
Our agents must not go rogue: prompt-injection fencing, fail-closed secret-scanning, DB-mutation guards, evidence guards, process-mining conformance. Table stakes for trusting the system.
Validate your AI-written product code
The QA system is the quality gate for AI-generated code: regression tuned to AI failure modes, behavioral locking during refactors, synthetic edge-case data, pre-merge validation gates. This is why you need the system at all.
Left-heavy coverage, full pyramid
Unit + integration depth first — where AI-written code actually goes wrong — then API, UI/E2E, and regression. Not another inverted, UI-only wedge.
Behavioral locking
Before an AI-assisted refactor, characterization suites lock the system's expected behavior — so a “cleaner” rewrite that changes 47.13 to 47.14 gets caught, not shipped.
Synthetic edge-case data
Generated boundary values, malformed input, injection strings, Unicode, timezone and rounding cases — the tests human authors skip.
Output-validation gates in CI
AI-generated changes don't merge until they clear the functional — and, as scoped, security and performance — boundaries.
Most QA tooling drives your app's UI and stops there. We build the whole gate — because we work in your repo, not just your browser.
We design into, build into, and maintain your stack — and you own it.
QA Agent Ops operates a thin control plane that designs into, builds into, and maintains your system in your repos, your environment, under your constraints. It isn't a diagram — the same production orchestrator runs live customer tickets for both reference engagements, with SDKs as swappable executors behind stable seams. The control plane is ours; the system is your asset.
- ▸All 10 layers, on YOUR SDK
- ▸In YOUR repos
- ▸YOUR environment + constraints
- ▸The asset you OWN
- ▸Design + build all 10 layers
- ▸Operate the fleet
- ▸Maintain — SDK / drift / reliability
- ▸Improve — OCPM loop, evolve
- ▸Harden — guardrails / secrets
Stays in place, stays yours
We build into and maintain your stack in your repos and your environment — never on Defkt infra. The control plane is ours; the system is your asset.
Constraints are first-class
Corporate firewall, provider restrictions, air-gap, read-only access — the control plane respects and enforces them, including scrubbing our tooling's traces off your host if policy demands it.
Work locally, push deliberately
Changes are authored and reviewed in the control plane, gated, then pushed through your own review process. We don't edit blind on the production host.
Reversible & attributed
Every change is version-controlled and commits under your identity and conventions — never Defkt's. Nothing is left uncommitted where a branch-switch could lose it.
Build it, or take over what you've got.
Two engagements, one constant: you own the platform; we engineer and operate it. Not ready for either? There's a third way in below.
Build it
You want an owned QA agent system on your stack — but building one that's actually reliable (guardrails, knowledge corpus, process-mining, a test suite of its own) is a multi-month senior-engineering effort. We design and build it, then run it.
Run, harden & improve what you have
An engineer built a harness; it's load-bearing but fragile — breaks on SDK bumps and firewall changes, under-hardened, no tests of its own, and the builder has moved on. We take ownership of its reliability and evolution.
Start with a Coverage & Gap Assessment
A ~2-week diagnostic of your whole test estate — unit / integration / API / UI / regression: depth, reliability, flakiness, coverage — plus a tooling and automation-maturity evaluation. You get a Current State & Gap Coverage Report and a 2-week / 1-month / 3-month roadmap.
$12,500 fixed — credited 100% toward a Build Engagement signed within 90 days. Worst case, it becomes your future hire's onboarding pack.
Ten layers — designed, built, and run.
Each layer is something we engineer for you — greenfield or completing what you started — then operate, maintain, and improve. The layers guard in two directions: our agents' conduct, and the code your AI writes. OCPM (L6) is the engine: it tells you whether a run was sound and drives the loop that makes the system better.
Testing workflow & orchestration
The agent QA pipeline (scout → plan → execute → review → playbook) and its state machine + stage gates, fit to how you ship — up to durable Temporal orchestration: replayable control flow, human-in-the-loop hints into a running agent, clean cancel, live progress.
Operate the fleet; keep the workflow sound as your app changes.
The agents
The roster — planner, executor, reviewer, troubleshooter, scoped read-only sub-agents — their prompts, reasoning patterns, and autonomy bounds. SDK-swappable executors: Claude and OpenAI Agents SDK run side-by-side in our production orchestrator.
Tune and retune as models and the app evolve.
Tools & integrations
The project toolkit/CLI (API-driven test-data setup, contract/DB helpers), synthetic edge-case data — boundary values, malformed input, injection strings, Unicode, timezone/rounding — browser automation, MCP tools, and Jira/Linear/GitHub wiring.
Maintain integrations through API + SDK churn.
Guardrails / hooks
Both directions: guards on our agents' conduct — fail-closed secret-scanning, injection fencing, tool/arg guards, evidence guards, DB-mutation guards — and guards on your AI-written code: AI-failure-tuned regression, behavioral locking during refactors.
Keep them fail-closed and current; add new ones as risks appear.
Domain-knowledge corpus
Page Object Models, the project knowledge base, a hybrid search index, and validated playbooks — produced by a corpus factory: grounding gates, then execution-feedback re-runs until the session converges clean. Only validated sessions enter the corpus.
Curate continuously; it gets smarter every ticket.
OCPM workflow mining
engineAn object-centric process-mining pipeline over every run — conformance, anti-pattern detection, best-practice scoring, audit-grade evidence — plus computer-vision verification of screenshot evidence and statistical drift detection between runs.
The audit → improvement loop: find gaps, fix, re-measure. The evaluation + observability layer most QA tooling doesn't have — and the engine of getting better.
Reliability layer
Retry/resume bridges, circuit breakers, crash detection + recovery, idle-window instrumentation.
Incident response on the environmental failure classes — firewall kills, dead daemons, crashes.
Security hardening
Secrets out of model context, passwordless / read-only DB roles, provider-trace scrubbing where your environment requires it.
Continuous hardening as the surface changes.
The system, under test
A real test suite for the QA system itself — unit, black-box, deterministic replay sims — with a CI coverage gate that only ratchets up.
Keep it green; every change ships with its tests.
CI/CD integration
Wire it into your pipeline — GitHub Actions / GitLab — trigger on PR / merge / ticket-status, gate merges, post results back, provision the test env per run. The merge gate doubles as the validation gate for AI-written code.
Keep triggers and gates healthy through CI + SDK churn.
Design, build, operate — then keep making it better.
An engineering + SRE partner for one high-value system, across its whole lifecycle. Severity model, SLA, incident response, crash recovery — not best-effort patching.
Coverage & Gap Assessment — optional
Not sure yet? A ~2-week diagnostic of your test estate across every layer → a Current State & Gap Coverage Report + a 3-horizon roadmap. The fee credits 100% toward a Build within 90 days.
Build Engagement or Stabilization Sprint
We either design + build the system (entry A) or assess + bring an existing one under control (entry B) — landing a tested, hardened, baselined system you own.
Steady-state
We maintain it, run it (Operate and up), and improve it. A monthly reliability + conformance report, proactive upgrade management, and incident response.
Improve
The OCPM audit loop and an evolution backlog keep raising coverage and reliability; a quarterly architecture review keeps the system the right shape.
You stay in control.Every change reaches your stack through your review process. Walk away any time — it's your asset; the system and its improvements are yours under the engagement terms.
SDK-agnostic — or fully custom.
The core scout + tester workflow runs on three SDKs in production today. We build against — and maintain against — seven normalized seams every modern agent SDK exposes, with a thin per-SDK adapter for the differences. Or we build fully custom, straight to the model. We've migrated a production harness between SDKs once already.
Cursor SDK
The deepest engagement — the full ten-layer stack, built and run end to end. Deepest quirk knowledge.
Claude Code / Agent SDK
Scout + tester proven in production — and one of the two production executors in our ticket orchestrator, running live customer tickets under durable (Temporal) orchestration.
OpenAI Codex / Agents SDK
Scout + tester proven in production — and the second production executor in our orchestrator: identical tool surface, transcripts normalized so one analysis + process-mining pipeline consumes both SDKs.
Pi agent + open-weight model
Our daily production driver internally — customized Pi instances run orchestration tooling, safety auditing, cross-agent integrations. The no-proprietary-vendor path: data sovereignty, cost, air-gap. The engagement adapter graduates on the first customer run.
Fully custom · direct-to-model
No vendor SDK — a bespoke agent wired straight to the model, through any router, or to a local open-weight endpoint. The control plane is the runtime; we own all seven seams. Choose it for control, routing, determinism, and data sovereignty — true metered cost, not token savings.
Two ways to run it — we model both and recommend
Cursor, Claude Code, and OpenAI Codex seats are vendor-subsidized — a flat seat bundles far more model usage than the equivalent metered API spend. Run your QA agents on the same SDK and they ride capacity you've already bought, at below-raw-API token economics. That directly shrinks the LLM-passthrough line.
A bespoke agent wired straight to the OpenAI / Anthropic / Google APIs, through any router (OpenRouter, LiteLLM, your gateway), or to local open-weight models. You pick this path for control, routing flexibility, determinism, and data sovereignty — not to save on tokens.
We're not religious about the runtime. We size the recommendation to your stack, the subscriptions you already hold, and your control requirements — the ten-layer build is the same either way.
Proven in production.
QA Agent Ops is productized from QA agent systems we design, build, and operate in production today — for regulated data-pipeline and enterprise GovTech SaaS teams (US, anonymized). Every number below maps to delivered work, not a demo.
workflow, agents, tools, guardrails, knowledge corpus, OCPM, reliability, hardening, self-test, CI/CD
scout + tester on Cursor, Claude Code, and Codex — Claude + OpenAI executors swap inside one production orchestrator
per-file 100% coverage on the logic layer, with a CI gate that only ratchets up
measured down from 100% after a guard fence on untrusted content
Two decades in high-reliability software, where “good enough” does not exist. The same rigor goes into every QA agent system we build, operate, and hand over.
The honest options are all bad.
QA Agent Ops is the missing option: an outside team that engineers and owns one specific system end to end — sized to the system, not to a headcount, and the asset stays yours.
Build + own it in-house
2–4 eng-months, then ~$15–22k/moGet: A system + an owner
Catch: Months to first value; bus factor; they'd rather build features.
Hire for the role — “AI Quality Architect” / “Chief Agentic Quality Architect”
$180–260k/yrGet: A dedicated owner (if you find one)
Catch: The market writes this JD under a half-dozen titles; the hunt runs 8–12 weeks for a unicorn — and the spec reads like our scope, because it is.
Stand up a toy agent and hope
cheapGet: A demo
Catch: Confidently-wrong results; no guardrails, no process-mining, no tests.
Buy a SaaS QA tool
$1–3k/mo + an operatorGet: A tool you don't own
Catch: Structurally UI/E2E-heavy — it drives your browser, not your repo; vendor lock-in; no custom domain knowledge; a new operator burden.
Outsource to embedded QA testers
$25–45/hr, scales with headcountGet: Rented human coverage, AI-assisted
Catch: Labor, not an asset — coverage scales linearly with hours, and the domain knowledge walks out at exit. Even the outsourcers rate their AI test output 70–85% accurate and keep humans in every loop.
Defkt QA Agent Ops
Build + $4.5–25k/moGet: An owned system, engineered and run
Catch: None we'll hide — Cursor / Claude / Codex proven; Pi + fully-custom builds supported.
What's real today vs. on the roadmap.
Cursor, Claude Code, and Codex are all production-proven in live customer projects — the core scout + tester workflow runs on all three. The deepest full ten-layer maturity is on the Cursor engagement; the others harden to full depth on the engagement that needs it. Pi + open-model is our daily internal driver — the engagement adapter graduates on the first customer run. We price and scope to the tier.
Sized to the system, not to a headcount.
Pick an entry — a 2-week assessment, a build, or a stabilization of what you have — then an ongoing retainer to run and improve it. An owned asset, not another subscription.
Coverage & Gap Assessment
~2-week diagnostic of your whole test estate (unit / integration / API / UI / regression — depth, reliability, flakiness, coverage) + tooling evaluation → a Current State & Gap Coverage Report and a 3-horizon roadmap. Worst case, it becomes your future hire's onboarding pack.
Build Engagement
Design + build the layers you need (L1–L10 as scoped, including CI/CD integration), then transition to a retainer.
Stabilization Sprint
Assess, bring-under-test, harden the top risks, stand up the control plane, establish a health baseline.
Care
- →SDK upgrade management
- →Drift + POM self-heal
- →Reliability incident response
- →Security upkeep
- →Monthly health + conformance (OCPM) report
Operate
- →Everything in Care
- →We run the fleet day-to-day
- →Trigger runs, triage failures, manage the queue
- →Curated knowledge stack
- →Active OCPM improvement loop
Evolve
- →Everything in Operate
- →Capability roadmap (new agents, coverage, integrations)
- →Self-evolution program
- →Quarterly architecture review
“+ LLM passthrough” is billed at cost. Running on a coding-agent seat you already pay for (Cursor / Claude Code / Codex) rides vendor-subsidized capacity, so this line stays small; a fully-custom direct-to-model build pays true metered token cost. We model both and recommend the one that fits your economics.
Traditional software costs 15–25%/yr of its build price to maintain. Agent systems run 50–120%/yr — token economics, model drift, weekly SDK churn, and continuous eval. The retainer isn't a markup; it's the real cost of keeping an agent system trustworthy. Budget it like traditional software and you under-fund it 3–5×.
Building this in-house is 2–4 senior-eng months ($40–90k) plus ~$15–22k/mo to keep an owner — and the market is now writing $180–260k “Chief Agentic Quality Architect” job specs for exactly this scope. The Build Engagement delivers the build for a project fee; Operate is that role, fractional, at roughly half a dedicated hire — a team's breadth, no ramp, no single-person bus factor.
Send us the shape of your stack.
Two paragraphs describing where you are is enough. We'll tell you in two paragraphs whether QA Agent Ops fits and at which entry point — or why it doesn't yet.
Same-business-day reply
Lisbon time — read by an engineer who can actually scope it, not a bot or a form.
A 30-minute technical call
Do you have a system to run and harden, or do you want one built? Which SDK, which environment, what constraints. No slide deck.
An honest fit diagnosis
A scoped proposal — Coverage & Gap Assessment, Build Engagement, or Stabilization Sprint — or why it's not a fit yet, and what would change that.
QA is our core product and deepest expertise — it's what we lead with. But the capability underneath — reliable agent systems with real guardrails, a domain-knowledge corpus, process-mining, and CI/CD integration — is domain-general. Have an adjacent agentic workflow that needs the same production-grade treatment? We can design, build, and operate that too, on request, under the same control-plane model. QA stays the main line.