Chat, search, and retrieval-augmented answers have moved from "interesting demo" to "a section of the backlog" for most of the products we work on. What's changed recently isn't the underlying models — it's that .NET finally has a proper abstraction layer for talking to them, instead of every team writing its own thin HTTP wrapper around a vendor SDK.

Key takeaways
  • Microsoft.Extensions.AI gives you a provider-agnostic IChatClient — the same code works against Azure OpenAI, OpenAI, or a local model via Ollama.
  • Retrieval-augmented generation (RAG) is the pattern behind most production AI features: embed your own content, retrieve the relevant pieces, and ground the model's answer in them.
  • Function calling lets the model call your own C# methods for real data instead of guessing — treat it like any other typed contract.
  • Model calls are an external dependency with real latency and failure modes. They need the same timeout, retry, and fallback discipline as any other outbound HTTP call.

A provider-agnostic starting point

Microsoft.Extensions.AI plays the same role for AI clients that ILogger plays for logging — a shared abstraction that any provider can plug into, so your application code isn't locked to one vendor's SDK shape.

csharp
builder.Services.AddChatClient(services =>
    new AzureOpenAIClient(
        new Uri(builder.Configuration["AzureOpenAI:Endpoint"]!),
        new DefaultAzureCredential())
    .AsChatClient("gpt-4o-mini"))
    .UseFunctionInvocation()
    .UseLogging();

Everything downstream is written against IChatClient, not against Azure OpenAI directly — swapping providers, or adding a local model for offline development, is a registration change, not a rewrite.

csharp
public class SupportAssistant(IChatClient chatClient)
{
    public async Task AnswerAsync(string question, CancellationToken ct)
    {
        var response = await chatClient.GetResponseAsync(
            [
                new ChatMessage(ChatRole.System, "You are a support assistant for astrivo customers. Be concise."),
                new ChatMessage(ChatRole.User, question)
            ],
            cancellationToken: ct);

        return response.Text;
    }
}

Grounding answers with retrieval-augmented generation

A model on its own only knows what it was trained on. RAG grounds it in your actual content: embed your documents into a vector store, retrieve the pieces relevant to the question, and include them in the prompt so the model answers from your data instead of guessing.

csharp
public class DocsRetriever(IEmbeddingGenerator> embedder, VectorStoreCollection store)
{
    public async Task> FindRelevantAsync(string question, CancellationToken ct)
    {
        var embedding = await embedder.GenerateAsync(question, cancellationToken: ct);

        var results = store.SearchAsync(embedding.Vector, top: 5, cancellationToken: ct);

        var chunks = new List();
        await foreach (var result in results)
            chunks.Add(result.Record);

        return chunks;
    }
}

The Microsoft.Extensions.VectorData abstraction works the same way across backends — Azure AI Search, Postgres with pgvector, Qdrant, or an in-memory store for local development — so the retrieval code doesn't change when the storage choice does.

Letting the model call your own code

Function calling turns the model into a router that can decide it needs real data and ask for it — rather than confidently inventing an answer. You define ordinary C# methods; the framework handles the back-and-forth.

csharp
[Description("Look up the current status of a customer's order by order ID.")]
static async Task GetOrderStatus(
    [Description("The order ID")] string orderId, IOrderService orders)
{
    var status = await orders.GetStatusAsync(orderId);
    return status?.ToString() ?? "Order not found";
}

var response = await chatClient.GetResponseAsync(
    messages,
    new ChatOptions { Tools = [AIFunctionFactory.Create(GetOrderStatus)] });

Treat the model call like any other dependency

Model APIs have latency and failure characteristics like any HTTP dependency — sometimes worse, since generation time varies with response length. The resilience patterns we already use for outbound HTTP calls apply directly: timeouts, retries with backoff, and a fallback response when the call fails or times out.

Operational noteLog prompts and completions for debugging, but be deliberate about what goes in them — strip PII and secrets before they reach a log sink, and track token usage per request. Cost and latency both scale with prompt size, and it's easy to lose track of both once a feature is live.

None of this requires betting the architecture on one vendor. The value of Microsoft.Extensions.AI is that AI features become another well-typed dependency in your DI container — testable, swappable, and observable the same way as everything else in the system.

Need help with this in production?

astrivo helps teams design, build, and modernize systems like this. Let's talk about your context.