What an AI application is
Strip away the demos and an AI application is a normal enterprise system with three new parts: a model you do not control, retrieval that feeds it context, and tools it is allowed to call. Everything around those three is engineering you already know: networks, databases, identity, integration, operations and cost.
That is why the map above has lenses. Pick one and the same system turns into a networking problem, a data problem or an identity problem, with the components that matter for that discipline lit up.
The shape that survives production
- Edge and API gateway own the way in: TLS, WAF, authentication and rate limits, with origins that accept traffic only from them.
- Agent decides what happens next. In most systems it is a supervisor with a small set of specialist workers, explicit stop conditions and checkpoints, so a run can resume after a crash.
- LLM gateway is the only way out to a model: routing, budgets, redaction, caching and one place to trace every call.
- Retrieval supplies the knowledge the model was never trained on, filtered at query time by who is asking.
- Ingestion keeps that knowledge fresh from the systems that own it, and copies their permissions onto every chunk.
- Tools are typed MCP servers with explicit permissions in front of real enterprise APIs. Writes carry idempotency keys, and the risky ones wait for a person.
Where teams get hurt
- Identity stops at the web app, so the agent calls tools as a shared service account and every user can reach whatever that account can.
- Retrieval leaks across teams, because permissions were not copied with the chunks or were not applied at query time.
- Observability arrives after the first incident, so nobody can say which retrieved text or tool call produced a bad answer.
- Evaluation lives in a spreadsheet instead of the release path, so a prompt change ships with no regression check.
- Retries meet side effects without idempotency keys, and an approval or a refund happens twice.
The guides attached to each component take these in order.
Where platforms meet
Every hop between two platforms is a seam. Each one needs identity federation (whose token crosses?), a private network path (no public egress), and a data decision (what leaves, in which region, logged where).
Azure and Databricks
- Identity
- Use Entra ID as the identity provider for both. Provision users and groups into Databricks with SCIM, map the Foundry managed identity to a Databricks service principal, and pass the user's token (audience: the Azure Databricks resource) wherever row-level access matters.
- Network
- Keep the hop private: Private Link to the workspace, and for serverless compute a network connectivity configuration (NCC) with private endpoints back to your storage and services. Turn off public network access on both sides.
- Data
- Decide what crosses. Send retrieved chunks and citations rather than tables. Keep both sides in one region for residency, and log the call on both (Log Analytics and Databricks system tables) so an audit can join them on a request ID.
Read moreYour agent should not be a superuserRelevant data can still be data the user may not seeShared agent memory is an access-control problem
Open source and Azure
- Identity
- Federate your OIDC provider with Entra ID, or use workload identity federation so self-hosted services get short-lived Entra tokens without stored secrets.
- Network
- Reach Azure services through private endpoints from your VNet, or over ExpressRoute or VPN from on-premises. Avoid public endpoints with API keys.
- Data
- Prompts and completions leave your boundary at this hop: check the region, retention and abuse-monitoring settings of the Azure model deployment, and log the request ID on your side.
Read moreWhat an AI gateway actually costs to runWhere your MCP server runs is a production decision
Open source and Databricks
- Identity
- Authenticate as a Databricks service principal with OAuth machine-to-machine tokens rather than personal access tokens, and federate users through the same IdP the workspace trusts.
- Network
- Front-end Private Link to the workspace, with IP access lists as the fallback where Private Link is not available.
- Data
- Query through SQL warehouses or Vector Search endpoints instead of copying tables out, so Unity Catalog lineage records who read what.
Read moreA vector store is infrastructure, not magic memoryA stale answer is a wrong answer
Every connection that can cross platforms (22)
Edge to Web App
The edge forwards to an app on another platform: lock the origin so it only accepts traffic from the edge.
Web App to Identity
Sign-in redirects across platforms: redirect URIs, cookie domains and the token audience all have to line up.
Web App to API Gateway
The browser session becomes an API call: the gateway validates a token issued for one platform by an API hosted on the other.
Identity to API Gateway
Tokens minted by one identity provider are validated by the other platform: issuer, audience and signing-key rotation must be trusted on both sides.
Read moreYour agent should not be a superuser
API Gateway to Agent
The request crosses from the gateway into the agent runtime: forward the user identity and a request ID along with the payload.
Agent to LLM Gateway
Every prompt crosses here. Budgets, redaction and the approved-model list are enforced on the gateway side, whichever platform runs the agent.
Read moreEvery autonomous run needs a budget for cost, time, actions and riskWhat an AI gateway actually costs to run
LLM Gateway to Model
Prompts leave for the model provider: the private endpoint, region and data-retention settings decide what crosses.
Read moreWhat an AI gateway actually costs to runTokenization quietly sets your cost, limits and reliability
Agent to Retrieval
The agent queries retrieval on another platform: the query must carry the end user, so per-document filters apply, and both sides log the same request ID.
Read moreRelevant data can still be data the user may not seeOne question can need many searchesThe retrieval cache that served stale policies
Retrieval to Vector Index
Retrieval reads an index hosted elsewhere: keep the query path private and the filter columns in step with source permissions.
Agent to Memory
Agent state is stored on another platform: partition it by user, encrypt it at rest, and set retention on both sides.
Read moreAgent memory is several stores, each with its own rulesShared agent memory is an access-control problem
Agent to Human review
Approvals wait on another platform: the paused run needs a durable callback, and the approver identity is recorded with the decision.
Read moreHuman review belongs where it has timing, authority and context
Agent to Tools
Tool calls cross platforms with real side effects: delegated tokens, idempotency keys and an allow-list per tool.
Read moreTool access is where AI becomes operational riskA retried agent job is a second chance to send the same emailPattern: the outbox for agent actions
Tools to Enterprise APIs
Tools reach systems of record over a private path with the delegated user token instead of a shared service account.
Read moreYour agent should not be a superuserWhen MCP earns its overhead, and when a direct API is better
Sources to Ingestion
Ingestion pulls from sources on another platform: use change feeds where you can, and copy source permissions with every document.
Read moreA stale answer is a wrong answerRetrieval quality is decided before anyone searches
Ingestion to Vector Index
Chunks and embeddings land in an index on the other side: version the embedding model with the index, so re-embedding is a deliberate migration.
Secrets to Agent
Credentials are fetched across platforms at runtime: use workload identity federation instead of exporting secrets.
Ingestion to Model
Ingestion calls the embedding model on another platform. Every chunk of every document crosses here, so it is the largest data flow and a real cost line: keep it private and in-region.
Read moreSimilar is not the same as correct: the limits of embeddingsTokenization quietly sets your cost, limits and reliability
Retrieval to Observability
Retrieval traces must reach tracing on the other platform: propagate W3C trace context so one request stays one trace across both.
Read moreIf you cannot trace it, you cannot run itEvaluate RAG in parts: retrieval, grounding, generation, citations
Identity to Retrieval
Users and groups must exist on the retrieval platform before filters can use them: automate provisioning with SCIM and map groups one to one.
Read moreRelevant data can still be data the user may not seeYour agent should not be a superuser
Sources to Audit
Lineage and access logs live on the data platform while audit lives on the other: export them on a schedule, so an audit can answer who read what.
Read moreEvery important AI claim needs a receiptGovernance is architecture, not a document after launch
Observability to Audit
Telemetry crosses into the security store: decide which fields are kept as audit evidence, and for how long.
Read moreGovernance is architecture, not a document after launch
LLM Gateway to Observability
Gateway spans flow to tracing on another platform: agree on one trace format, including token and cost attributes.
Node by node
Your network
Web App
Frontend3 linked
Identity
SSO · OBO2 linked
Understand
- ExplainerYour agent should not be a superuser
Operate
API Gateway
AuthN · limits4 linked
Secrets
Vault · KMS1 linked
Operate
Agent
Orchestration40 linked
Understand
- Deep diveThe hard part of building an agent is not making it act. It is making it stop.
- Deep diveSix ways to wire agents together, and the same three things break every time
- Deep diveAdding a second agent does not add intelligence, it adds a contract
- Deep diveMost agent controls do not actually control anything
- Deep diveEvery question your AI readiness review asks was answered months ago
- 24 more on the map
Implement
Decide
- Decision noteWe built our own TypeScript orchestrator, and stayed on it
- ComparisonChatbot, workflow or agent: choose by how much autonomy the task can bear
- ComparisonMost teams reaching for multiple agents do not need them
- ComparisonMost agents are workflows wearing a costume
- ComparisonLangGraph vs writing the agent loop yourself
Use
- ToolLangGraph
Reference
- ReferenceBuilding Effective Agents
LLM Gateway
Route · budget11 linked
Understand
- Deep diveMost agent controls do not actually control anything
- ExplainerTokenization quietly sets your cost, limits and reliability
- ExplainerGood guardrails make safe progress possible, not just block
- ExplainerEvery autonomous run needs a budget for cost, time, actions and risk
- ExplainerSafety is an architecture decision, not a moderation setting
- 6 more on the map
Observability
Traces · evals19 linked
Understand
- Deep diveEvery question your AI readiness review asks was answered months ago
- ExplainerEvaluate RAG in parts: retrieval, grounding, generation, citations
- ExplainerIf you cannot trace it, you cannot run it
- ExplainerEvaluate the workflow users see, not just the model
- ExplainerThe 3am agent run that nobody is watching
- 12 more on the map
Use
- ToolOpenTelemetry
Reference
- ReferenceOpenTelemetry Documentation
Tools
MCP servers20 linked
Understand
- ExplainerStructured output turns model text into something software can trust
- ExplainerProduction knowledge lives in systems of record, not only in PDFs
- ExplainerTool access is where AI becomes operational risk
- ExplainerA handoff is a contract: state, evidence, open questions, an owner
- ExplainerRetrieved content is data, never instructions
- 9 more on the map
Decide
Operate
Use
- ToolInstructor
Reference
- ReferenceModel Context Protocol
- ReferenceOWASP Top 10 for LLM Applications
Human review
Approval queue4 linked
Understand
Implement
Private subnet
Ingestion
Chunk · embed12 linked
Understand
- Deep diveNo model can reason about evidence your retriever never returned
- Deep diveProduction RAG does not fail loudly, and that is the whole problem
- ExplainerRetrieval quality is decided before anyone searches
- ExplainerChunk boundaries decide what your RAG system can know
- ExplainerSimilar is not the same as correct: the limits of embeddings
- 5 more on the map
Implement
Vector Index
Embeddings · hybrid7 linked
Understand
- Deep diveNo model can reason about evidence your retriever never returned
- Deep diveMost RAG failures are faithful answers to bad evidence
- ExplainerSimilar is not the same as correct: the limits of embeddings
- ExplainerA vector store is infrastructure, not magic memory
- ExplainerGood RAG needs the precise snippet and the context around it
- 1 more on the map
Retrieval
Hybrid · rerank29 linked
Understand
- Deep diveNo model can reason about evidence your retriever never returned
- Deep diveMost RAG failures are faithful answers to bad evidence
- Deep diveProduction RAG does not fail loudly, and that is the whole problem
- Deep diveEverything that makes an AI demo impressive is a constraint you have not added yet
- ExplainerConfidence is not evidence
- 16 more on the map
Implement
Decide
Operate
- ChecklistRAG production-readiness checklist
- Failure storyThe retrieval cache that served stale policies
Use
- ToolLlamaIndex
- Toolpgvector
- ToolRagas
Reference
Memory
Session · long-term7 linked
Understand
- Deep diveAdding a second agent does not add intelligence, it adds a contract
- ExplainerAgent memory is several stores, each with its own rules
- ExplainerShared state needs structure, attribution and versions
- ExplainerShared agent memory is an access-control problem
- ExplainerA handoff is a contract: state, evidence, open questions, an owner
- 2 more on the map
Systems of record
Sources
Docs · tickets · DBs6 linked
Understand
- Deep diveProduction RAG does not fail loudly, and that is the whole problem
- ExplainerRetrieval quality is decided before anyone searches
- ExplainerProduction knowledge lives in systems of record, not only in PDFs
- ExplainerA stale answer is a wrong answer
- ExplainerYour vector index does not know who is allowed to read it
Operate
- ChecklistWhen to bring in a compliance review
Model provider
Model
Provider API14 linked
Understand
- Deep diveYour AI system's biggest problem is probably not the model
- ExplainerModel choice is rarely your first production problem
- ExplainerLLMs generate one token at a time, so design for it
- ExplainerTokenization quietly sets your cost, limits and reliability
- ExplainerMore context can make an AI system worse
- 4 more on the map
Implement
Operate
- ChecklistWhen to bring in a compliance review
Use
- ToolLiteLLM
- ToolGuardrails
Reference
- ReferenceAttention Is All You Need
Security scope
Audit
Every call7 linked
Understand
- ExplainerAgents need arbitration rules before they disagree
- ExplainerGovernance is architecture, not a document after launch
- ExplainerYou cannot instruct a model into data protection
- Architecture patternPattern: the outbox for agent actions
Operate
On-premises
Enterprise APIs
Any system of record2 linked
Understand
Decide