This is the comparison most .NET developers are confused about when they first look at local inference, and it is a fair confusion to have. Microsoft.SemanticKernel is the Microsoft-maintained framework for orchestrating LLM-powered applications in .NET — prompts, planners, plugins, function calling, agent loops. llmdot is a runtime that runs a local GGUF model. The two come up in the same breath, but they live at different layers of the stack. They are complementary in the cases where both are appropriate, not competing.
Here is the honest framing.
What each one is
Microsoft.SemanticKernel is an orchestration framework. It defines the abstractions developers compose around — kernels, prompt templates, plugins, function calling, memory, planners. It connects to model backends through provider integrations: OpenAI, Azure OpenAI, ONNX runtime, Ollama, and others through community-maintained connectors. The model itself is not Semantic Kernel’s concern; the orchestration around it is.
llmdot is a model runtime. It loads a GGUF file from disk and executes a transformer through one of four architecture-agnostic templates. It exposes streaming token generation through IAsyncEnumerable<T> and supports the small bits of orchestration that belong at the runtime level — chat template formatting from GGUF metadata, sampling configuration, conversation history through ChatSession. It does not provide planners, plugins, or higher-level agent abstractions. That is not the job.
The cleanest way to say it: Semantic Kernel is the kitchen. llmdot is one of the appliances.
How they connect
The connection point is Microsoft.Extensions.AI. Semantic Kernel composes against IChatClient and the related abstractions; llmdot ships Llmdot.Extensions.AI, which exposes the runtime as an IChatClient through LlmdotChatClient. The DI registration is services.AddLlmdot(...). From Semantic Kernel’s perspective, llmdot is a backend like any other — you orchestrate, llmdot executes.
In practice, an architecture that uses both looks like this. Semantic Kernel handles the prompt templating, function-calling protocol, agent loop, memory, and planner. The model invocations underneath route to LlmdotChatClient, which loads a GGUF file from disk, formats messages through the chat template parsed from GGUF metadata, runs the model on the CPU (or on an optional Metal or Vulkan backend if a backend package is installed), and streams tokens back through IAsyncEnumerable<T>. The whole thing runs in-process. No sidecar service. No cloud round-trip.
Where the comparison gets useful
The comparison is most useful for the developer who has heard “use Semantic Kernel for local AI in .NET” and is trying to figure out what that actually means in practice. The answer is that Semantic Kernel handles orchestration; you still need a backend that runs the model. The choices for that backend, broadly, are:
| Backend | Where the model runs | Connector situation |
|---|---|---|
| OpenAI / Azure OpenAI | Cloud API | First-party SK connectors |
| ONNX runtime / Phi-3 ONNX | Local, requires ONNX conversion | First-party SK connector |
| Ollama | Local, sidecar HTTP service | Community SK connector |
LLamaSharp | Local, in-process via native llama.cpp | Community SK connector |
llmdot | Local, in-process, pure managed | Llmdot.Extensions.AI exposes IChatClient |
If you want local, in-process, no native toolchain, and a GGUF file you downloaded from the community catalog, llmdot is one credible answer to the “what backend” question while leaving Semantic Kernel free to do the orchestration work it is designed for.
What llmdot does not do
We want to be explicit about the scope boundary, because pretending otherwise would be unhelpful.
llmdot does not provide a planner. It does not provide a plugin abstraction. It does not provide a function-calling protocol layer. It does not provide memory or vector store integration. It does not provide an agent loop. It does not provide retrieval-augmented generation primitives. All of those are orchestration concerns and they live above the runtime.
What llmdot provides is: load a GGUF file, configure sampling, format a chat template, generate text, stream tokens, surface model capabilities (architecture, template, attention type, MoE status, sliding window, softcapping), expose all of that through idiomatic .NET APIs and IChatClient. Where you take it from there is where Semantic Kernel (or any other orchestration layer) comes in.
When you might pick one without the other
If you are building a small focused tool — a desktop summarizer, a CLI that drafts text from a local model, an embedded assistant inside an existing .NET application — the orchestration layer may be overkill. The llmdot APIs are enough on their own. LoadAsync, CreateChatSession, StreamAsync, done.
If you are building an agent application with planners, function calling, multiple tools, memory, and retrieval, the orchestration layer is what you want. The model backend is a configuration choice underneath. Pick a backend that fits your deployment constraints — llmdot if local-in-process matters; OpenAI if cloud is fine; ONNX if you have already committed to that conversion pipeline.
If you are doing both — an agent application that needs to stay fully local — the combination is the point. Semantic Kernel up top, llmdot underneath.
What we are not claiming
We are not claiming llmdot replaces Semantic Kernel. We are not claiming Semantic Kernel replaces llmdot. We are not claiming the orchestration layer should be in the runtime. We are claiming the layers are distinct, the integration point is IChatClient, and the comparison is mostly useful for clarifying which problem belongs where.
If you came here looking for “should I pick A or B?”, the answer is probably both, at different layers. If you came here looking for what a local-in-process backend for Semantic Kernel looks like in pure managed .NET, that is exactly the case llmdot is designed for.