Everything llmdot gives a .NET app.
A GGUF-native runtime that stays managed by default and reaches for the GPU only when you ask.
The managed runtime
GGUF-native loading
Memory-mapped GGUF ingestion compatible with the llama.cpp ecosystem. Read a community model from disk, decode its tensors, and run a forward pass — no conversion, no proprietary packaging.
Four execution templates
LLaMA-like, GPT-NeoX-like, Gemma-like, and LFM2-like. Every 1–8B family we care about collapses into one of them, parameterised by a TransformerConfig resolved from GGUF metadata.
CPU-first backend
The headline path is pure managed compute — no native toolchain, no per-RID binaries. It runs anywhere .NET runs.
Sampling
Greedy, temperature, top-k, top-p, and repetition control — the decoding knobs you expect, exposed through a small, typed API.
Tokenizer
In-process tokenization across BPE variants, driven by the tokenizer metadata embedded in the GGUF file.
The .NET you already write
IChatClient / Microsoft.Extensions.AI
Llmdot.Extensions.AI implements IChatClient so llmdot drops into the Microsoft.Extensions.AI abstraction — swap a cloud client for a local one without rewriting your app.
Streaming with IAsyncEnumerable<T>
Tokens stream through IAsyncEnumerable, the idiom .NET already uses for async sequences. No callbacks, no custom event model.
DI & Hosting
Register the model with Microsoft.Extensions.DependencyInjection and Microsoft.Extensions.Hosting — the same wiring as the rest of your services.
Trimming, NativeAOT & single-file
A pure managed core means the default install stays trimming-friendly, NativeAOT-friendly, and single-file-publish-friendly.
Acceleration & modality, additive
Vulkan backend (planned)
Llmdot.Backends.Vulkan offloads individual operations to the GPU on Linux and Windows — incremental, not an all-or-nothing graph rewrite.
Metal backend (planned)
Llmdot.Backends.Metal targets Apple Silicon through the same per-operation IComputeBackend contract; CPU stays the fallback.
Vision / multimodal (planned)
Llmdot.Multimodal.Vision adds a SigLIP2 vision encoder and connector on top of the base LLM backbone. The core runtime is unchanged.
Ready to load your first model?
The quickstart walks you from dotnet add package to streamed tokens.