Frequently asked questions.
Straight answers about llmdot's scope, status, and design.
What is llmdot?
llmdot is a .NET-native runtime for local GGUF language model inference. It loads community GGUF models directly and runs decoder-only and hybrid architectures in the 1–8B range through four architecture-agnostic execution templates. The default install is pure managed .NET.
Is llmdot production-ready?
No. llmdot is pre-alpha. The specification, architecture, and execution-template design are stable, but the implementation is in active development. Do not use it in production yet.
Which .NET versions does it support?
Target frameworks are net8.0, net9.0, and net10.0, with net8.0 LTS as the compatibility floor.
Which models and architectures work?
Quantized GGUF models in the 1–8B parameter range that map to one of four templates: LLaMA-like (llama, phi3, qwen2, stablelm, mistral), GPT-NeoX-like (gptneox, phi2), Gemma-like (gemma, gemma2), and LFM2-like (lfm2, lfm2_moe). Multimodal vision variants are planned.
Does llmdot need a native toolchain?
No. By default llmdot is pure managed .NET with zero native runtime dependencies — no P/Invoke to a native library, no per-RID binaries, no ONNX conversion. It runs anywhere .NET runs.
Does it support GPU acceleration?
Optionally, and additively. Planned Vulkan and Metal backends offload individual operations to the GPU through an IComputeBackend contract while CPU stays the default. There is no all-or-nothing graph rewrite.
How is llmdot different from LLamaSharp?
Both run local GGUF models in .NET, but they make opposite choices on the native-versus-managed axis. LLamaSharp binds to native llama.cpp; llmdot is managed by default, trading some peak performance for packaging and deployment simplicity. See the comparison for detail.
How does it relate to Semantic Kernel and Microsoft.Extensions.AI?
Different layers. Semantic Kernel orchestrates; llmdot executes. llmdot's connection point is IChatClient via Llmdot.Extensions.AI, so it plugs into the Microsoft.Extensions.AI abstraction that SK and other tools build on.
Is llmdot free and open source?
Yes. llmdot is MIT-licensed and developed in the open by Cognisoc.
Still stuck? Get in touch, read the docs, or open an issue on GitHub.