llmdot
Quickstart

From package to streamed tokens.

Five steps to run a local GGUF model in your .NET app — no native toolchain, no ONNX conversion.

Pre-alpha. The code below reflects the target API shape. Track current implementation status in the roadmap.

  1. Add the package

    Target net8.0, net9.0, or net10.0. The core install is pure managed .NET.

    # in your project directory
dotnet add package Llmdot.Core

  2. Get a GGUF model

    Grab a quantized 1–8B model in GGUF format — for example phi-3-mini-q4_k_m.gguf. No conversion step, no proprietary packaging.

  3. Load the model

    llmdot reads the GGUF metadata, resolves the right execution template, and builds the model graph.

    using Llmdot;

await using var model = await LlmModel.LoadAsync("phi-3-mini-q4_k_m.gguf");

  4. Create a chat session and stream

    Tokens arrive through IAsyncEnumerable<string> — iterate them with await foreach.

    await using var session = model.CreateChatSession();

await foreach (var token in session.StreamAsync(
    "Summarise the GGUF format in three sentences."))
{
    Console.Write(token);
}

  5. Integrate with Microsoft.Extensions.AI

    Add Llmdot.Extensions.AI to expose the model as an IChatClient — the same abstraction your cloud clients implement.

    dotnet add package Llmdot.Extensions.AI

IChatClient client = model.AsChatClient();
// register it in DI and inject it like any other IChatClient

Next: read the architecture to see how the execution templates work, browse the full feature list, or check the FAQ. Full reference lives in the docs.