We chose Azure AI Search for retrieval because ingestion and filtering stay simple

Context

Our AI application answers from the organisation's documents and cites them. Three forces decided where retrieval should live: ingestion already runs in Databricks, queries need filtering, and different use cases need the same documents ranked differently.

Why Azure AI Search

Ingestion stays one pipeline. Our Databricks ingestion pipelines feed the index directly. Configuring a new source is a pipeline change, not a new system to run.

Filtering is simple. Fields marked filterable narrow a query in the same call, so scoping by source, date or use case happens inside retrieval rather than after it.

Several semantic configurations on one index. Each use case gets its own semantic configuration, naming the fields the ranker treats as title, content and keywords, while the documents are indexed once. A query picks its configuration at request time.

How the product handles permissions

Azure AI Search supports permission-aware retrieval through what Microsoft calls security trimming: store the groups allowed to read each document in a filterable field, and pass the user's groups as a filter on every query, so a query can only return what that user may see. For vector queries the filter runs before the nearest-neighbour search by default, so a document the user may not see never takes a place in the top results.

The other benefits

• Hybrid search is built in. Keyword (BM25) and vector results are merged by reciprocal rank fusion, so exact terms such as product codes and error strings still match when embeddings blur them.
• The semantic ranker re-scores the top 50 results with Microsoft's language models before they reach the prompt, and can return captions that make citations easier to show.
• Facets and scoring profiles cover browsing and boosting, which vector similarity alone does not.
• It is a managed service. Replicas scale queries and partitions scale size, independently, and nobody on the team runs a search cluster.

What it costs

The index schema is a contract. Adding a field is cheap, but changing how an existing field is analysed or filtered means building a new index, so it pays to plan schema changes like API versions and cut over in one switch.

Cost follows search units, replicas times partitions, and the semantic ranker is billed on top, so both are worth watching as usage grows.

When we would revisit

If the documents came to live entirely in the lakehouse, and retrieval there covered hybrid ranking, filtering and semantic configurations as well, keeping a second store in step would stop paying for itself.