Enterprise AI Architecture Checklist: 12 Gates Before Build
A strong AI architecture does not start with a model, vector store or agent framework.
It starts with a traceable chain from business task → data → decisions → system boundaries → controls → operations.
The best starting question is therefore not “which AI platform do we need?” but:
What evidence must exist before this architecture is safe, maintainable and economically sensible to build?
Gartner's 2026 enterprise-architecture research increasingly positions EA as a coordinating function for AI transformation, technology choices and safe, scalable AI systems. Sources: Gartner – Coordinate, Rationalize and Accelerate Enterprise AI Value, Gartner – Lead AI-Driven Business and Technology Transformation.
The short version: 12 architecture gates
| Gate | Core question | | --- | --- | | 1. Business Outcome | Which concrete task or decision improves? | | 2. Workflow | Where does AI sit in the real process? | | 3. Data | Which data is actually required? | | 4. Identity & Permissions | Who may see and trigger what? | | 5. Retrieval | Which information must be retrieved rather than generated? | | 6. Model | Which model capabilities are truly needed? | | 7. Agent / Tools | Must AI act, or is a controlled workflow enough? | | 8. Integration / API | Which systems remain systems of record? | | 9. Security & Privacy | Which boundaries and data flows are critical? | | 10. Evaluation | How will quality become measurable? | | 11. Observability | How are failures, drift, cost and latency exposed? | | 12. Operations & Ownership | Who operates, decides and responds? |
A gate is satisfied when there is evidence, not merely an opinion.
Gate 1 – Business outcome
Core question
Which concrete work becomes better, faster, cheaper or more reliable?
Weak requirements:
- “We want GenAI.”
- “We need a copilot.”
- “We want agents.”
Stronger:
“Support employees need reliable answers from approved technical documentation with citations, without manually searching several systems.”
Required evidence
- target user,
- current workflow,
- pain or friction,
- expected outcome,
- measurable baseline.
Typical failure mode
The team optimises model quality even though the business workflow has no meaningful bottleneck.
Gate 2 – Workflow
Core question
Where exactly does AI sit in the process, and who decides the next step?
Distinguish between:
- assistive AI,
- a bounded LLM step,
- deterministic workflow,
- bounded agent,
- higher-autonomy agent.
If the process is stable, an agent is often unnecessary.
See AI Agent vs Workflow vs Automation.
Required evidence
- process steps,
- inputs/outputs,
- human gates,
- failure paths,
- allowed actions.
Typical failure mode
A flexible agent replaces a process that could be simpler and more reliable as a workflow.
Gate 3 – Data
Core question
Which data is required, and in what quality?
Relevant considerations include:
- ownership,
- freshness,
- document identity,
- versions,
- structured vs unstructured data,
- deletions,
- data quality.
In RAG, the data pipeline is part of product behaviour.
Deep dive: Connect RAG data sources.
Required evidence
- source inventory,
- data classification,
- change frequency,
- authoritative source,
- known quality issues.
Typical failure mode
The demo dataset is clean while production data is not.
Gate 4 – Identity & permissions
Core question
Which identity applies at each system boundary?
User identity may affect:
- retrieval,
- API calls,
- tool calls,
- actions,
- tenant isolation,
- audit.
“One service account for everything” is rarely a robust enterprise model.
Required evidence
- identity provider,
- roles/groups,
- ACL model,
- delegation,
- service identities,
- tenant boundaries.
For RAG: Permissions & security trimming.
Typical failure mode
The system returns technically correct content from a source the user was not authorised to access.
Gate 5 – Retrieval
Core question
Which information must be found at runtime?
Not every enterprise question belongs in RAG.
Common retrieval use cases include:
- document knowledge,
- policies,
- manuals,
- wikis,
- knowledge bases.
Exact live values such as inventory, account balances or order status often belong behind a controlled API instead.
Required evidence
- query types,
- source of truth,
- freshness requirement,
- relevant document types,
- retrieval baseline.
Test retrieval against realistic questions: RAG Evaluation.
Typical failure mode
Everything is copied into a vector store even when structured live data should be queried directly.
Gate 6 – Model
Core question
Which model capability is actually required?
Consider:
- language quality,
- reasoning,
- structured outputs,
- tool calling,
- context length,
- multimodal capability,
- latency,
- cost,
- hosting.
Avoid coupling business logic unnecessarily to a single model.
Gartner's 2026 EA briefing argues that AI architecture should account for model-agnostic design and multiprovider strategies rather than treating AI only as an innovation initiative. Source: Gartner – Compute, Controls and Cost Define AI’s New Architectural Reality.
Required evidence
- realistic test cases,
- quality comparison,
- latency/cost profile,
- data/hosting requirements.
Typical failure mode
The “best” model is selected before the system's required capability is understood.
Gate 7 – Agent / tools
Core question
Does the model actually need discretion over actions?
Tool calling changes the risk profile.
A system that only generates text has different failure modes from one that:
- changes records,
- sends messages,
- closes tickets,
- calls APIs,
- prepares payments.
Required evidence
- permitted tools,
- tool identities,
- argument boundaries,
- read/write scope,
- human gates,
- stop conditions.
For existing agents: AI Agent Evaluation.
Typical failure mode
“Tool calling works” is confused with “tool calling is controlled”.
Gate 8 – Integration / API
Core question
Which system remains authoritative for each piece of information or action?
AI should not create uncontrolled copies of systems of record.
An API-first architecture separates:
- AI orchestration,
- retrieval,
- business logic,
- system APIs,
- UI.
Required evidence
- system-of-record matrix,
- API contracts,
- read/write boundaries,
- idempotency,
- error handling.
Typical failure mode
Business logic moves into prompts even though it belongs in deterministic services or workflows.
Gate 9 – Security & privacy
Core question
Which data and actions may cross which system boundaries?
Review:
- data classes,
- processing locations,
- logs,
- model providers,
- retention,
- secrets,
- tool permissions,
- prompt-injection paths,
- exfiltration paths.
Self-hosting or EU hosting does not automatically make a use case GDPR-compliant. The actual processing matters.
See GDPR-compliant AI for enterprises and Private AI vs public cloud.
This technical guidance does not constitute legal advice.
Required evidence
- data-flow diagram,
- data classes,
- provider/hosting model,
- logging design,
- security boundaries.
Typical failure mode
Architecture reviews where the model is hosted but ignore data flowing through connectors, logs or tools.
Gate 10 – Evaluation
Core question
What does “works” mean?
Evaluation should begin before implementation.
Depending on the system:
- correctness,
- groundedness,
- retrieval quality,
- citation accuracy,
- task success,
- tool accuracy,
- policy compliance,
- human escalation.
Required evidence
- golden dataset,
- baseline,
- critical failure classes,
- go/no-go criteria.
See RAG Evaluation and AI Agent Evaluation.
Typical failure mode
Project members like the demo, but nobody can measure regressions.
Gate 11 – Observability
Core question
How will we know that the system is getting worse?
An AI system needs more than infrastructure monitoring.
Depending on the use case, monitor:
- end-to-end latency,
- model/tool failures,
- retrieval failures,
- data freshness,
- cost per task,
- no-answer rate,
- escalations,
- regressions.
Required evidence
- events/traces,
- metrics,
- alerting,
- sampling/review process,
- incident diagnosis.
Typical failure mode
Servers and APIs are “green” while answer quality or retrieval silently degrades.
Gate 12 – Operations & ownership
Core question
Who owns the system after the pilot?
Define owners for:
- product/use case,
- data,
- models,
- infrastructure,
- security,
- evaluation,
- incident response.
Required evidence
- owner per layer,
- runbook,
- change process,
- rollback,
- review cadence,
- cost accountability.
Typical failure mode
The pilot team dissolves and nobody owns the production AI system.
Minimum architecture vs target architecture
A pilot does not need the complete target platform.
It must, however, test the critical architecture drivers realistically.
Minimum architecture
Example:
User → Identity → Application → Retrieval/API → Model → Response + Logging
Target architecture
Later additions may include:
- multiple models,
- agents/tools,
- additional sources,
- policy layer,
- observability,
- admin/evaluation UI,
- cost controls,
- multi-tenancy.
The pilot can simplify infrastructure. It should not artificially remove the central risks.
See Plan an AI pilot.
A useful reference model
A technology-neutral enterprise AI architecture can be thought of as:
User / System
↓
Identity & Policy
↓
Application / Workflow
↓
AI Orchestration
↙︎ ↓ ↘︎
RAG / Search · Tools / APIs · Models
↓
Evaluation / Observability / Audit
↓
Operations & Ownership
The products may change. The boundaries should remain understandable.
Architecture Decision Records for AI
Important architecture decisions should not live only in slide decks.
For each material decision, document:
- decision,
- context,
- options,
- drivers,
- risks,
- consequences,
- exit/revisit trigger.
Examples:
- Why hybrid search?
- Why public cloud rather than on-premise?
- Why workflow rather than agent?
- Why model abstraction?
- Why this permission strategy?
This reduces “architecture by memory”.
12-gate scorecard
A simple review can mark each gate:
- Green: evidence exists and the decision is robust.
- Yellow: assumption is plausible but needs validation in the pilot.
- Red: critical question remains open or the architecture is inconsistent.
A go-live does not require perfection. But critical red gates around identity, security, evaluation or operations are not cosmetic backlog items.
Anti-patterns
Tool-first architecture
“We bought Azure/OpenAI/LangChain – now we need a use case.”
Prompt as business logic
Deterministic rules move into unversioned prompts.
One model everywhere
One model is used for every function without task-level evaluation.
RAG everything
Structured live data is unnecessarily forced into document retrieval.
Agent everything
Autonomy is introduced where a workflow would be easier to control.
Pilot without a production path
The demo works only with test data, admin rights and manual intervention.
When an architecture review makes sense
A review is particularly useful when:
- several AI pilots are emerging in parallel,
- platform or cloud decisions are pending,
- RAG and agents are moving into production systems,
- security/identity is becoming an afterthought,
- vendor lock-in is increasing,
- nobody can articulate a target architecture.
Next step
If you are planning an AI initiative and still deciding between platforms, RAG, agents, private AI or integration approaches, the architecture should become defensible before a larger build.
I help enterprises turn business goals, system boundaries, data, identity, integration, evaluation and operating model into a pragmatic target architecture: AI consulting & solution architecture.
If a PoC already exists and the question is instead “is this system production-ready?”, see AI Production Readiness.