Enterprise AI Architecture Checklist: 12 Gates Before Build

Production readiness is not a component check

An architecture diagram shows components and connections. It does not prove that an AI solution can run safely with real data, real load and real failures. A successful demo run proves little more. Guardrails are only a claim until they have been shown to work at the system boundaries that matter.

Production readiness does not come from having every architectural building block in place. It comes from being able to assess critical decisions about data, access, security, quality, operations, failure handling, cost and exit against defensible evidence.

This memo is therefore not a tool checklist. It asks whether a specific solution is sufficiently constrained, observable, operable and recoverable for its intended use.

Before the gates: scope and failure budget

No production-readiness review is defensible without a clear scope. Before reviewing the technical gates, establish:

• Which concrete process or outcome will the solution support in production?
• Which users or systems will use it?
• Which actions may it perform, and which must it never perform?
• What is explicitly out of scope?
• Which error would be commercially unacceptable?

A support assistant that proposes a cited answer has a different failure budget from an agent that closes tickets or changes customer data. Scope defines which failures may safely escalate and where the architecture must stop. Without that boundary, a team can approve a technically neat solution for an unclear use case.

How to assess each architecture gate

Each gate receives an explicit status:

• GO: The relevant decision has been made, evidence exists, and the remaining risk is within accepted bounds.
• MITIGATION NEEDED: There is no immediate blocker; the remaining risk is explicitly assessed and accepted. The measure, owner, deadline and verification criterion are documented. Whether mitigation may happen before or after go-live depends on the specific risk.
• BLOCKED: A central prerequisite is missing, or the risk cannot currently be bounded or assessed sufficiently. Do not approve production.

Production readiness review

1. 01

   Set the scope

   Which process, users, actions and unacceptable failures are in this review?

   Review boundary documented

2. 02

   Assess gate evidence

   Which decision is evidenced and which still rests on an assumption?

   GO, mitigation or blocker

3. 03

   Isolate blockers

   Which system boundary or failure class prevents a responsible approval?

   Risk stated precisely

4. 04

   Define mitigation

   Which measure reduces the risk, who owns it and how will it be verified?

   Owner and verification

5. 05

   Document the decision

   Which trade-offs are deliberately accepted and when will the decision be revisited?

   GO, MITIGATION NEEDED or BLOCKED documented

Neither a numeric total nor an averaged traffic light replaces this decision. One genuine blocker in access control, security boundaries, evaluation or recovery must not be offset by eight green gates.

Gate 1: Data — provenance, freshness and reproducibility

Architecture decision

Which data may influence the output, which source remains authoritative, and how can the data state at the time be understood? A retrieval index, cache or derived dataset is not a second business truth.

Required evidence

• an authoritative source, business identity and provenance for every relevant data class,
• versions, validity status and freshness requirements,
• rules for update, delete, archive and restore,
• known data-quality limits,
• a traceable path to explain or reproduce an output from the data state at the time.

Typical blockers

Unknown data provenance, stale content without visible freshness, deletes that are not propagated, or a data pipeline whose result cannot be reproduced.

Mitigation and GO

A pilot can start with fewer sources. It still needs a clear authoritative source, documented freshness boundaries and verifiable lifecycle events. GO is defensible when a critical output can be traced to its source, version and point in time. Connecting RAG data sources covers lifecycle and ingestion in depth; for document-centric systems, see connecting a DMS to RAG and connecting SharePoint to RAG.

Gate 2: Identity & access — effective permissions at every system boundary

Architecture decision

Which identity applies throughout the execution path: at login, retrieval, API calls and tool actions? What matters is the effective permission at the end of that path, not merely a correct frontend login.

Required evidence

• identity mapping between application, data source and external APIs,
• roles, ACLs, tenant boundaries and service identities,
• permitted read and write actions per tool,
• least-privilege boundaries and traceable delegation,
• tests for permission changes and denied access.

Typical blockers

Post-filtering sensitive content, missing identity mapping or tool calls through an over-privileged service account are not acceptable loose ends.

Mitigation and GO

A read-only assistant can begin with less delegation than an active agent. GO is only defensible when unauthorised evidence cannot reach model context and every action uses the correct identity with the minimum scope. RAG permissions and security trimming covers the retrieval perspective.

Gate 3: Security — containment, not retrospective control

Architecture decision

Where are the trust boundaries, and which inputs, tools and outputs may cross them? In AI systems, detection alone is insufficient: critical actions need technical limits.

Required evidence

• documented trust boundaries for user input, context, models, tools and egress,
• controls against prompt and context injection,
• secret handling that keeps secrets out of prompts, clients and unprotected logs,
• tool isolation, sandboxing, rate limits and action limits,
• safe defaults for unknown, invalid or unexpected model results.

Typical blockers

A free-form model result can control privileged tools, a connector can send data out without control, or a secret is reachable through a prompt, log or tool output.

Mitigation and GO

Read-only tools, strict schemas, allow-lists, explicit approvals and technical egress boundaries reduce risk. GO is defensible when a manipulated model result cannot trigger an action beyond its bounded permission. Privacy, data residency and the right operating model are a separate adjacent review: GDPR-compliant AI for enterprises.

Gate 4: Evaluation — measurable quality before and after change

Architecture decision

What does “works” mean for this use case, and which failure class is unacceptable? Model, prompt, retrieval, data and tool behaviour must remain testable against the same controlled baseline.

Required evidence

• quality criteria and go-live acceptance criteria,
• a versioned golden dataset or regression set with realistic cases,
• expected failure classes, negative cases and permitted escalations,
• task success and, where relevant, tool success rather than merely plausible answers,
• a process that tests changes for regressions before rollout.

Typical blockers

“It works in my tests”, no reproducible baseline, no acceptance criteria, or changes without regression testing are insufficient for approval.

Mitigation and GO

A small, curated set of critical cases is better than a large, vague benchmark collection. GO is defensible when known critical failures are measurable and a change cannot silently degrade cases that were already solved. RAG evaluation: golden datasets and regression tests goes deeper on metrics and failure diagnosis. For agentic workflows with tool calls or state changes, this system-wide approval is not sufficient on its own: the specific execution path also needs evaluation. AI Agent Evaluation covers that work.

Gate 5: Observability — make failures, drift and cost diagnosable

Architecture decision

Which signals make a poor output, drift or expensive request traceable to its cause? Infrastructure monitoring alone is insufficient when model, retrieval and tool calls jointly produce an outcome.

Required evidence

• request and trace correlation across model, retrieval and tool steps,
• separate signals for latency, token and tool cost, errors and escalations,
• retrieval and quality indicators that match the defined failure classes,
• data-minimising logging rules, access to diagnostic data and retention,
• a practical incident-diagnosis path.

Typical blockers

Failures are visible only through user complaints, a wrong answer cannot be assigned to a component, or cost and latency are not visible per request or task.

Mitigation and GO

Do not log everything. Capture the minimum evidence required to distinguish causes. GO is defensible when the team can trace a representative failure to the request, affected steps, data state and version without recording unnecessary sensitive content.

Gate 6: Failure modes — timeouts, tool failures, fallbacks and recovery

Architecture decision

How does the system behave outside the happy path? The happy path demonstrates functionality; failure paths demonstrate production readiness.

Required evidence

• expected handling for model outage, rate limit, timeout and unavailable retrieval,
• rules for failed tools, partial execution and duplicate execution,
• retry strategy, backoff, and idempotency keys or deduplication for appropriate mutating actions,
• status reconciliation when execution state is unknown,
• fail-closed or fail-open decisions, fallback and human handoff,
• recovery and reversal strategy for critical actions.

Typical blockers

A failed tool call can leave an unclear partial state. A timeout or connection loss on a mutating action does not mean that the action was not executed; a simple retry can execute it twice.

Mitigation and GO

For irreversible actions, human gates or explicit confirmation are often safer than autonomy. If an unknown execution state cannot be resolved automatically, the system needs a human handoff or safe escalation. GO is defensible when critical failure paths have been deliberately tested and uncertainty leaves the system in a safe, explainable state.

Gate 7: Cost & capacity — load, limits and economic boundaries

Architecture decision

What real load must the solution support, and how does it constrain cost and degradation when models, retrieval or APIs are scarce?

Required evidence

• assumptions about users, concurrency, peak load and expected context size,
• provider rate limits and behaviour when they are exceeded,
• cost per task including model, retrieval and tool calls,
• budget limits, alerting, backpressure or queueing,
• a degradation strategy, such as shorter output, deferred processing or safe escalation.

Typical blockers

Unknown unit economics, no defined behaviour for rate limits, cost that can escalate without limit, or no defensible assumption about real load.

Mitigation and GO

An early version may set conservative limits and deliberately queue complex work. GO is defensible when cost per relevant task is known and the solution degrades predictably under peaks rather than becoming uncontrolledly expensive or unreliable.

Gate 8: Vendor lock-in & exit — portability, export and rebuild

Architecture decision

Is the dependency consciously accepted, and is a realistic exit possible if price, quality, availability or strategy changes? Lock-in is not inherently bad; unexamined lock-in is an operating risk.

Required evidence

• provider-specific APIs, vectors, stores and agent runtimes are named clearly,
• exportable data formats and ownership of data, configurations and evaluation cases,
• versioned prompts, workflows and decisive infrastructure configuration,
• ability to rebuild derived data and critical system state,
• documented change and revisit triggers.

Typical blockers

A provider change is only theoretically possible, data or evaluation knowledge cannot be exported, or the architecture decision exists only in the memory of individual project members.

Mitigation and GO

For deliberately accepted trade-offs, record an ADR with context, options, decision, risks, consequences, and exit and revisit triggers. GO is defensible when the cost of dependency is known and a change or rebuild can be described as a concrete process. Private AI vs public cloud examines operating-model choices in more detail.

Gate 9: Operations — ownership, runbooks, change and rollback

Architecture decision

Who responds on Monday morning when it stops working? Production needs a named owner, an incident path and a safe change chain, including for model, provider and data changes.

Required evidence

• owners for product, data, technical platform, security and approval of critical changes,
• an incident path, runbooks and a reachable escalation route,
• version management for model, prompt, data and tool configuration,
• deployment, rollback, backup and restore procedures,
• regular re-evaluation when providers, models, data or usage change.

Typical blockers

The pilot team dissolves, no one owns the runbook, a model change cannot be rolled back, or an incident has no clear first responder.

Mitigation and GO

A lean runbook is enough when it covers the real failure paths. GO is defensible when ownership, first response and recovery for a critical incident can be followed in practice.

What a pilot may simplify — and what it must not

A pilot may be smaller: scaling, convenience, UI, degree of automation and selected integrations can remain deliberately limited.

It must not rest on assumptions that production will have to discard entirely. The identity model, fundamental data and ACL logic, evaluability, failure boundaries, ownership and irreversible exit decisions therefore belong in the pilot early.

A pilot may be smaller. It should not hide the risks that will determine production.

The review output: decisions, mitigations and blockers

An architecture review does not end with an abstract score dashboard. Its output is a decision artefact:

• GO decisions with their supporting evidence,
• MITIGATION NEEDED with measure, owner, deadline and verification criterion,
• BLOCKED with the concrete risk, missing prerequisite and condition for unblocking,
• ADRs for deliberately accepted trade-offs,
• revisit triggers for changes to data, models, providers, load or risk profile.

This keeps production readiness from becoming a one-time assessment. It can be reviewed again when relevant change occurs, rather than leaving decisions in slides or the memory of the project team.

Four anti-patterns before go-live

Tool-first architecture

“We bought a platform; now we need a use case.” Without scope and failure budget, there is no defensible approval boundary.

Prompt as business logic

Deterministic rules move into unversioned prompts. That makes behaviour, testing and rollback unnecessarily fragile.

Agent everything

Autonomy is introduced where a bounded workflow with clear handoffs would be safer and easier to recover.

Pilot without a production path

The demo works with test data, admin permissions and manual intervention. The critical production risks have not been tested; they have been hidden.

When an architecture review is needed before production

A review is especially useful when a PoC is moving into production, RAG or agents will access real enterprise systems, critical tool actions are planned, or model, provider and platform decisions will be expensive to reverse later.

For a new initiative with open system boundaries, target architecture or a build plan, AI consulting and solution architecture is the relevant starting point. If a system, pilot, RAG or agent setup already exists before rollout or go-live, AI Production Readiness is the appropriate alternative for checking these gates against its real state.