Kit · Tools
Tools and radar
Every product on the map, and the libraries I reach for around it. The rings are my own experience in real systems, not an industry ranking.
Radar
Adopt 5
I use it in production and would again.
Trial 4
In a real project, not yet proven at scale.
Hold 0
Tried it; would not start a new project with it.
Nothing in this ring yet.
Quadrants: AI and data · Platform · Security and identity · Integration and operations
The list
Open a row for where it fits, where it does not and what to use instead. Map links open the component with that stack selected.
- 1InstructorStructured outputs for LLMs using Pydantic models.Adopt AI & ModelsAny stack
- Fits
- Typed outputs with retries in a few lines.
- Does not fit
- Streaming-heavy UIs where partial objects matter.
- Alternatives
- Native structured outputs
pip install instructorHomepage - 2LangGraphBuild stateful, multi-agent applications as explicit graphs.Adopt AI & ModelsOpen source
- Fits
- Explicit graphs when routing logic is the hard part.
- Does not fit
- Simple single-agent apps; the ceremony is not worth it.
- Enterprise concerns
- Checkpointer backends and state size need an operational owner.
- Alternatives
- OpenAI Agents SDK, Semantic Kernel
pip install langgraphHomepageOn the map: Agent
- 3pgvectorOpen-source vector similarity search for PostgreSQL.Adopt Data & DatabasesOpen source
- Fits
- Keeping embeddings next to relational data for most workloads.
- Does not fit
- Hundreds of millions of vectors with strict latency targets.
- Enterprise concerns
- Index tuning (HNSW parameters) needs a DBA conversation.
- Alternatives
- Qdrant, Azure AI Search
On the map: Vector Index
- 4OpenTelemetryObservability for modern systems, with GenAI semantic conventions.Adopt Observability & OpsOpen source
- Fits
- One tracing standard across agents, tools and HTTP.
- Does not fit
- Nothing; adopt it even for a prototype.
- Enterprise concerns
- GenAI semantic conventions are still stabilising; expect attribute renames.
- Alternatives
- Vendor SDKs
pip install opentelemetry-sdkHomepageOn the map: Observability
- 5RagasEvaluation framework for retrieval-augmented generation pipelines.Adopt Observability & OpsAny stack
- Fits
- Retrieval and faithfulness metrics in CI against a golden set.
- Does not fit
- Metrics without a golden set; it will not invent ground truth.
- Enterprise concerns
- LLM-as-judge cost on every CI run; sample.
- Alternatives
- DeepEval, Custom evals
pip install ragasHomepage - 6LlamaIndexData framework for connecting LLMs to your documents and data.Trial Data & DatabasesOpen source
- Fits
- Fast ingestion and retriever experiments over many document types.
- Does not fit
- Teams that already own a retrieval pipeline; it adds a second abstraction.
- Enterprise concerns
- Version drift between integrations; pin aggressively.
- Alternatives
- LangChain, Hand-rolled with pgvector
pip install llama-indexHomepageOn the map: Retrieval
- 7OpenAI Agents SDKLightweight framework for multi-agent workflows with handoffs and guardrails.Trial AI & ModelsAny stack
- Fits
- Smallest API for supervisor and hand-off patterns with built-in tracing.
- Does not fit
- Provider-neutral platforms; it leans OpenAI.
- Enterprise concerns
- Vendor coupling in orchestration code.
- Alternatives
- LangGraph
pip install openai-agentsHomepage - 8Unseen UIAccessible React components for AI interfaces, by StackUnseen.Trial AI & ModelsAny stack
- Fits
- Chat, tool calls, approvals and streaming as controlled React components.
- Does not fit
- Non-React stacks.
- Enterprise concerns
- Alpha releases; the API still moves between versions.
- Alternatives
- assistant-ui, Vercel AI Elements
npm install @stackunseen/uiHomepage - 9LiteLLMUnified LLM API interface across providers.Trial NetworkingOpen source
- Fits
- Provider abstraction plus a proxy with cost tracking and rate limits.
- Does not fit
- Single-provider teams; it is one more hop.
- Enterprise concerns
- The proxy is in your data path; treat it as production infrastructure.
- Alternatives
- Portkey, Direct SDKs
pip install litellmHomepageOn the map: LLM Gateway
- 10GuardrailsValidate and control LLM outputs with reusable validators.Assess SecurityAny stack
- Fits
- Turning output policies into testable validators.
- Does not fit
- Latency-sensitive paths; validators add round trips.
- Enterprise concerns
- Validator hub supply chain; review what you install.
- Alternatives
- Instructor + Pydantic, Custom validators
pip install guardrails-aiHomepage
On the map, not rated yet43 products I have not put a ring on
- ·AI Agent FrameworkMosaic AI Agent Framework: agents authored in code, evaluated and served next to the data, traced with MLflow.Not rated AI & ModelsDatabricks
On the map: Agent
- ·AI Search indexersPull from Blob, SQL and SharePoint with built-in skills for chunking and vectorising.Not rated Data & DatabasesAzure
On the map: Ingestion
- ·AI Vector SearchMosaic AI Vector Search. Delta Sync indexes follow their source tables through Change Data Feed. Unity Catalog governs who can query the index, not per-document source ACLs: copy those into filter columns.Not rated Data & DatabasesDatabricks
On the map: Vector Index
- ·Airflow · UnstructuredScheduled pipelines for parsing, chunking and embedding; full control of chunking.Not rated Data & DatabasesOpen source
On the map: Ingestion
- ·Azure AI SearchHybrid search and semantic ranker built in, document-level security filters, indexers that pull from Azure sources.Not rated Data & DatabasesAzure
On the map: Vector Index
- ·Azure OpenAIOpenAI models in your tenant and region, with private endpoints.Not rated AI & ModelsAzure
On the map: Model
- ·Cosmos DBPer-user partitions for threads and memory, globally distributed.Not rated Data & DatabasesAzure
On the map: Memory
- ·Declarative PipelinesLakeflow Connect pulls from SaaS sources such as SharePoint; Lakeflow Declarative Pipelines (formerly DLT) chunk and embed; Change Data Feed keeps indexes current.Not rated Data & DatabasesDatabricks
On the map: Ingestion
- ·Delta · Unity CatalogGoverned tables and volumes; permissions travel with the data.Not rated Data & DatabasesDatabricks
On the map: Sources
- ·Foundry Agent ServiceAzure AI Foundry Agent Service: a managed runtime with threads, tools and Entra-scoped connections.Not rated AI & ModelsAzure
On the map: Agent
- ·LakebaseManaged Postgres next to the lakehouse for agent state and history.Not rated Data & DatabasesDatabricks
On the map: Memory
- ·Logic Apps · TeamsApproval cards in Teams that resume the workflow when answered.Not rated AI & ModelsAzure
On the map: Human review
- ·Model ServingFoundation Model APIs plus external models, governed through the AI Gateway.Not rated AI & ModelsDatabricks
On the map: Model
- ·Postgres · S3 · wikiWhatever already holds the knowledge; connectors are yours to write.Not rated Data & DatabasesOpen source
On the map: Sources
- ·Redis · PostgresRedis for working state, Postgres for durable history and facts.Not rated Data & DatabasesOpen source
On the map: Memory
- ·Semantic rankerHybrid queries with Microsoft's reranker and security trimming in one call.Not rated AI & ModelsAzure
On the map: Retrieval
- ·Slack · Jira approvalApprovals land where reviewers already work; the agent waits on a callback.Not rated AI & ModelsOpen source
On the map: Human review
- ·Vector Search hybridHybrid keyword and vector queries with reranking. Per-document access comes from filter columns populated at ingestion, applied for the calling identity.Not rated AI & ModelsDatabricks
On the map: Retrieval
- ·vLLM · open weightsYour GPUs, your data residency; capacity planning becomes your problem.Not rated AI & ModelsOpen source
On the map: Model
- ·API ManagementPolicies for JWT validation, quotas and versioning; it can also act as an AI gateway.Not rated NetworkingAzure
On the map: API Gateway
- ·APIM AI gatewayToken-based limits, semantic caching and load balancing across Azure OpenAI deployments.Not rated NetworkingAzure
On the map: LLM Gateway
- ·Container AppsScale-to-zero containers with built-in auth integration to Entra ID.Not rated Compute & InfraAzure
On the map: Web App
- ·Databricks AppsApps hosted next to the data, with workspace identity and Unity Catalog access built in.Not rated Compute & InfraDatabricks
On the map: Web App
- ·Front Door · WAFGlobal edge, managed WAF rule sets and bot protection in one service.Not rated NetworkingAzure
On the map: Edge
- ·Kong · EnvoyPlugins for auth, rate limits and request logging; runs anywhere.Not rated NetworkingOpen source
On the map: API Gateway
- ·Mosaic AI GatewayRate limits, usage tracking, guardrails and payload logging on every serving endpoint.Not rated NetworkingDatabricks
On the map: LLM Gateway
- ·Next.js on K8sContainerised frontend; streaming responses need proxies that do not buffer SSE.Not rated Compute & InfraOpen source
On the map: Web App
- ·Nginx · ModSecurityReverse proxy with OWASP CRS rules. You own the rule tuning and the DDoS response.Not rated NetworkingOpen source
On the map: Edge
- ·Entra ID · OBOFirst-class on-behalf-of flow, so downstream Graph and API calls carry the user.Not rated Identity & AccessAzure
On the map: Identity
- ·HashiCorp VaultDynamic, short-lived credentials; the strongest model, and a service you must run.Not rated SecurityOpen source
On the map: Secrets
- ·Key VaultSecrets and keys read by managed identity, so config holds no credentials.Not rated SecurityAzure
On the map: Secrets
- ·KeycloakOIDC provider with token exchange; on-behalf-of flows are possible but you configure them.Not rated Identity & AccessOpen source
On the map: Identity
- ·OpenSearch · LokiAppend-only log store; immutability and retention are your configuration.Not rated SecurityOpen source
On the map: Audit
- ·Purview auditPurview audit and Log Analytics hold the record; Sentinel is the SIEM that correlates it with sign-ins.Not rated SecurityAzure
On the map: Audit
- ·Secret scopesWorkspace secrets, optionally backed by Azure Key Vault.Not rated SecurityDatabricks
On the map: Secrets
- ·System tablesAudit, lineage and access logs queryable as tables in Unity Catalog.Not rated SecurityDatabricks
On the map: Audit
- ·Workspace SSO · OAuthUsers and service principals from your IdP; permissions resolve in Unity Catalog.Not rated Identity & AccessDatabricks
On the map: Identity
- ·App InsightsOpenTelemetry-native; AI Foundry adds tracing and evaluation views.Not rated Observability & OpsAzure
On the map: Observability
- ·MCP servers on K8sOne server per system of record, deployed and scaled like any service.Not rated IntegrationOpen source
On the map: Tools
- ·MCP via APIMExpose existing APIs as MCP tools behind API Management policies.Not rated IntegrationAzure
On the map: Tools
- ·MLflow TracingTraces, evaluation runs and judges in the same place as the model versions.Not rated Observability & OpsDatabricks
On the map: Observability
- ·UC functions · MCPUnity Catalog functions and managed MCP servers as governed tools.Not rated IntegrationDatabricks
On the map: Tools