Back to writing
4 min readAI Security

What AI Agent Assurance Claims Actually Cover

How to read an agent assurance claim through its scope, assessor, test version, and renewal conditions.

By Kevin O'Connor

An agent vendor can show you a legitimate certificate and still leave your most important access question unanswered. The certificate might concern the company's AI management system. Your question might be whether its support agent can read another customer's attachments. Those are different objects of assessment.

I start with the decision the buyer needs to make: which identity may perform which action on which resource, under which conditions? Then I look for assurance materials that actually address that decision. A logo alone doesn't tell me whether the deployed tool permissions, tenant boundaries, or approval path were examined.

I am affiliated with TKOResearch, which offers AgentBoundary assessments. That relationship matters here. I include its published rules as one scoped program, and I haven't independently audited the providers in this comparison or verified a customer certificate. The program descriptions below were reviewed on September 9, 2026.

Read the object of assurance first

ISO/IEC 42001:2023, edition 1 specifies requirements for an AI management system. An organization can use that system to govern development and operation of agents. Certification against it does not, by itself, establish that a particular agent rejects unauthorized tool calls.

A SOC 2 examination concerns controls at a service organization. Its usefulness depends on the system description, categories and controls in scope, testing, exceptions, and customer responsibilities. Type 1 concerns a specified date; Type 2 adds operating effectiveness over a period. AICPA's report review checklist explicitly distinguishes those periods and asks about gaps.

A technical robustness assessment can get closer to the access question, but only where its scenarios exercise the relevant boundary. A prompt-injection score from a read-only assistant says little about an agent that can issue refunds using a privileged service account.

A bounded program comparison

This table compares the types of assurance available, not provider quality. Each provider link is a primary description of its own program.

ProgramScope and methodIndependence and verificationVersion and renewal limits
BSI ISO/IEC 42001 certificationAudit of an organization's AI management system; the optional readiness assessment is separate from certification.BSI describes independent certification. Check the issuing body, accredited scope, and certificate status through BSI's verification resources.Standard: ISO/IEC 42001:2023. BSI's general system-certification rules describe a three-year term, surveillance at least once or twice yearly, and recertification. Confirm the applicable audit program on the actual certificate.
ERM CVS ISO 42001 certificationStage 1 reviews governance documentation; Stage 2 assesses its operation.ERM CVS describes an independent certification decision and says it separates internal audits from its certification relationships. Obtain the certificate and issuer confirmation; this page is not a certificate registry result.Uses the 2023 standard, planned surveillance, and a three-year cycle. The certified organizational scope must include the activity you rely on.
TKOResearch AgentBoundaryScoped adversarial assessment of agent flows using a published scenario suite and execution-oriented scoring.Provider describes third-party testing. My affiliation prevents treating this article as an independent endorsement. Verify the issued result and its publication status.Published offer describes a 12-month verification listing and annual or change-triggered reassessment. Request the exact suite revision, models, prompts, tools, permissions, and scoring configuration.

The table deliberately doesn't translate management-system certification into an agent access pass. BSI and ERM CVS describe organizational assurance programs. AgentBoundary describes a narrower technical program. Neither category should silently inherit the other's meaning.

The AgentBoundary scope rules identify model, prompt, memory, tool, permission, architecture, and retrieval changes that can invalidate applicability. A buyer should make those changes visible in its own release process. An anniversary date is insufficient when a new connector changes what the agent can reach next Tuesday.

Follow one permission through the report

Consider a hypothetical purchasing agent allowed to draft an order but unable to approve payment. I would ask the supplier to point to the test where a retrieved document instructed the agent to approve its own order. I would then inspect the actual control path: tool selection, authenticated subject, order ownership, approval identity, and downstream payment result.

The useful materials would identify the tested build, show whether the request reached the payment service, and explain where rejection occurred. A model refusal is informative, but a payment API denial tied to the correct principal answers a different and stronger enforcement question. If the model later changes its answer, that API boundary should still hold.

I would also look for ordinary successful cases. A test package that blocks everything hasn't established a usable purchasing workflow. Include an authorized draft, an approved payment, an expired approval, a changed amount, and a different customer's order. The report should show both intended use and boundary failures without pooling them into one reassuring percentage.

Make assurance expire when its assumptions expire

Keep the assurance reference next to the deployed configuration. Record the report date and scope, relevant version identifiers, exceptions accepted by your organization, and the owner who decides whether a change needs retesting. Treat missing access scenarios as open questions, even if the supplier has strong organizational certification.

NIST AI RMF 1.0 can help organize that ongoing risk work. It is voluntary guidance, not a universal agent-access certification scheme.

For procurement, the practical request is a short mapping from each consequential permission to its enforcement point and supporting assessment. That gives the buyer something it can review when access expands, a model changes, or a certificate remains valid while the system underneath it has moved on.

Email updates

Get new research by email

In-depth notes on AI security, threat research, and practical defensive work.

To unsubscribe, email kevin@kevinbytes.com.