Open source · C# on .NET 10 · by Managed Code

Run LLMs
like a fly brain.

Synapse runs large language models on your own machine, in pure C#. No Python, no server, no network. And it is designed to fire only the regions each token needs.

  • 190.6tokens/s on the local CPU
  • 3.2×faster short request
  • 2.3×less memory

Recorded local measurements · MacBook Pro M2 Pro · Qwen2.5 0.5B · median of 5 runs · how we measured

Scroll to see the idea

01The problem How engines work today

Every token fires the whole model.

LLM engines run every block for every token and keep every weight in memory. A wall of compute, whether the token needs it or not.

02The idea Next

A wave, not a wall.

A fly's brain sends a signal through only the few regions it needs (FlyWire connectome). Synapse does the same with a model: dense attention always runs, the model's own router picks the expert, and a region is skipped only with proof.

03Memory Built · layer drop

Cold regions stay on disk.

A layer a profile drops never loads: its weights stay on disk and it gets no KV cache. Its quality cost is measured before it is dropped.

240 MiBper dropped layer, Qwen2.5-7B

04Scale Built · wiring in next

One model. Several machines.

Regions run on different machines. Only the hidden state travels between them.

3.5 KiBper token for Qwen2.5 0.5B

Latest measurements · GitHub Actions

How does Synapse compare?
Pick a model. See the difference.

Green is Synapse. Choose a model, a runner class and an answer length. Each chart names its unit and timing scope; missing measurements stay visible. These short diagnostic runs do not establish a quality or release winner.

Results load from the published JSON. Open the links above to inspect the original runs.

Tests

No run loaded.

Performance

No run loaded.

Synapse vs Microsoft Foundry Local

Foundry Local runs downloaded ONNX models on the runner. This is local inference, not the Azure AI Foundry cloud service. Compare the same model family below; GGUF and ONNX weights, threads and cache policies differ.

Waiting for measured evidence.

Synapse CPU and MLX on Apple GPU

Current hosted measurements use different devices and weights. For Synapse Metal vs llama.cpp Metal on the same local Mac, see the Apple GPU charts below.

Waiting for measured evidence.

Recorded local measurements · September 2026 · Apple M2 Pro

Our models on Apple silicon.
Synapse beside the other engines.

These earlier local diagnostics use different hardware from GitHub Actions above. The runnable decoder models shown here are Qwen2.5 0.5B and Qwen2.5 7B. Green is Synapse, blue is llama.cpp, purple is MLX. Check the answer-quality note as well as speed.

Writing speed

190.6tokens/s, Synapse

  • Synapse190.6
  • llama.cpp167.2
  • LLamaSharp147.0
  • dotLLM31.7

Model: Qwen2.5 0.5B · Q8_0 · tokens/s · higher is better · Synapse reported decode vs llama.cpp native evaluation: phase definitions differ

Whole short request

3.2×faster than llama.cpp

  • Synapse216 ms
  • llama.cpp690 ms
  • LLamaSharp945 ms
  • dotLLM2,227 ms

start to answer, 2 threads · lower is better

Peak memory

2.3×less than llama.cpp

  • Synapse555 MiB
  • llama.cpp1,256 MiB
  • LLamaSharp1,275 MiB
  • dotLLM1,190 MiB

whole process · lower is better

MacBook Pro, Apple M2 Pro, 32 GB. Qwen2.5 0.5B Instruct Q8_0, prompt “The capital of France is”, 8-token answer. Median of 5 runs after 3 warm-ups; all four engines wrote the same 8 tokens. Diagnostic runs, not the final 30-run release gate. All runs and raw JSON ↗

Qwen2.5 0.5B · Apple M2 Pro GPU · September 2026

Tokens, memory and speed. From 4k to 32k.

Synapse and llama.cpp use the same Q8_0 file and selected KV precision. MLX uses its own 8-bit weights. Memory is peak physical footprint; it differs from the peak RSS charts above. Correct answers and matching token IDs appear below the charts.

Pick a context and a KV cache

Memory

  • Synapse534 MiB
  • llama.cpp535 MiB
  • MLX13,500 MiB

peak footprint · KV cache 384 MiB

First token

  • Synapse27.63 s
  • llama.cpp26.62 s
  • MLX15.51 s

time to first token · lower is better

Full answer

  • Synapse27.82 s
  • llama.cpp26.81 s
  • MLX17.62 s

Synapse and llama.cpp stop after 19 tokens, MLX writes 128

Writing speed

  • Synapse92.2
  • llama.cpp92.9
  • MLX60.1

tokens per second · higher is better

Same tokens: Synapse wrote the same tokens llama.cpp wrote. Right answer: Synapse 2/2, llama.cpp 2/2, MLX 0/2.

MacBook Pro, Apple M2 Pro 19-core GPU. Qwen2.5 0.5B Q8_0, one prompt per context: repository text with a hidden number, then “state the number and summarize”, up to 128 tokens. Every engine gets the same text and its token IDs are checked. Median of 2 runs after a warm-up, one engine at a time. MLX runs its own 8-bit weights and cache format; Synapse and llama.cpp also map the 644 MiB model file. Perplexity on identical tokens matches llama.cpp within 0.01% at 4k–32k. Run T and raw JSON ↗

A big window, small memory

7×less than llama.cpp

  • Synapse234 MiB
  • llama.cpp1,645 MiB

peak footprint · 131k window, 4k prompt, FP16 KV · the KV cache grows with use instead of being reserved

Second question, same document

0.13 sto the first token

  • Reuse prefix0.13 s
  • Read it again23.8 s

30,000-token document · Metal, FP16 KV · the same answer either way

One run per row, a fresh process each; diagnostics, not a paired verdict. Runs U and V ↗

Qwen2.5-7B-Instruct-1M · Apple M2 Pro GPU · 29 September 2026

One model. Two engines.
Compare the same weight format.

Peak memory

4,241MiB, Synapse Q4_K_M

  • Synapse4,241 MiB
  • llama.cpp4,849 MiB

peak RSS, mapped weights included · median of 2 alternating rounds · lower is better

Writing speed

17.8tokens/s, Synapse Q4_K_M

  • Synapse17.8
  • llama.cpp28.2

tokens/s · median of 2 alternating rounds · higher is better · Synapse reported decode vs llama.cpp native evaluation: phase definitions differ

Small draft, big check

1.35×faster, same output

  • + 0.5B draft23.5
  • 7B alone17.4

tokens per second, Q8_0, chat prompt · the 0.5B drafts 3 tokens, the 7B checks them in one pass, 54% kept

Drop layers, never load them

−475 MiB2 of 28 layers dropped, perplexity −1.1%

  • All 28 layers7,255 MiB · 12.16
  • 2 dropped6,780 MiB · −1.1%
  • 4 dropped6,326 MiB · +39%
  • 8 dropped5,381 MiB · +137%

peak memory · perplexity on the test text · a dropped layer's weights are never mapped for the GPU and its KV is never allocated · only the 2-layer drop passes the quality gate

Apple M2 Pro 19-core GPU, Qwen2.5-7B-Instruct-1M. The two selected-format engines use the same GGUF file, 3,528-token prompt, 128-token limit and FP16 KV; medians of 2 alternating fresh-process rounds. Exact text parity was checked separately on the pinned 16-token “The capital of France is” prompt; it is not a quality verdict for the long answer. Q4_K_M perplexity is 11.93 vs 12.16 for Q8_0 on 2,048 scored tokens. Diagnostics, not a release verdict. 7B measurements and quality source ↗

Not there yet. We say so.

  • Prompt processingThe local 0.5B first-token charts show the long-prompt gap. For 7B, Synapse records 16.9–17.3 s to its first token; llama.cpp records 10.8–11.2 s of native prompt evaluation. Those 7B phase definitions differ, so they do not establish a first-token speedup.
  • Q4 writing speedThe 7B in Q4_K_M uses less peak RSS than llama.cpp in these two rounds. Synapse reports 17.4–18.2 decode tokens/s; llama.cpp reports 27.0–29.3 native-evaluation tokens/s. The phase definitions differ; faster Q4 kernels are next.
  • Past 32kThe 7B-1M model and head size 128 kernels run now. Windows past 32k have not been measured yet.
  • Smart context on CPU onlyReading only the relevant KV pages costs +3.4% perplexity while reading 30% of an 8k context. The GPU kernels are next.
  • CUDA untestedMetal runs. The CUDA backend is written but has not run on an NVIDIA GPU yet; it rejects Q4_K_M files for now.
  • One familyOnly Qwen2 runs today. More model families and MoE expert paging are next.

Why Synapse

Built for .NET, from the kernels up.

C#

Pure C#, no Python

.NET 10 owns the portable path. Rust takes a kernel only after profiling proves it is worth it.

⌂

Local first

One process, no server, no network. The cluster is an option, never a requirement.

=

Same answer as llama.cpp

Qwen2.5 runs fully in C#. In the benchmark it writes exactly the same tokens as llama.cpp.

≈

Small drafts, exact answer

A 0.5B model drafts tokens and the 7B checks them in one pass. The answer stays exactly the 7B's, 1.35× faster on chat text.

!

Fails loudly

Model math, precision and cache formats are explicit. An unsupported fast path fails instead of guessing.

✓

Verified and cross-platform

Model downloads are verified. Tests and benchmarks run on macOS, Linux and Windows.

Architecture

Decide what, how and where. Then run the wave.

  1. 1

    Model IR

    What can run.

  2. 2

    Execution IR

    How, and how precisely.

  3. 3

    Deployment plan

    Where each region runs.

  4. 4

    Wave scheduler

    Runs only the regions a token needs.

Every worker runs the same runtime

C# kernelsRust kernelsMetalCUDA · untestedWeight residencyHot KV cacheZoneTree state

Local

An app uses Synapse in one process. Works without Orleans and without a network.

Cluster Planned

An Orleans control plane coordinates workers. Workers pass the hidden state directly to each other.

Read the architecture

Roadmap

18% → 102% of llama.cpp. And climbing.

First version18%
SIMD C# kernels88%
Rust kernels102%

CPU writing speed as a share of llama.cpp · the line is llama.cpp

Done

  • Qwen2.5 runs fully in C#, same answer as llama.cpp
  • Apple GPU (Metal), up to 32k context
  • Tokenizer with the same IDs as llama.cpp
  • Long-context quality checks
  • Model graph and verifier
  • Benchmarks on three operating systems
  • Qwen2.5-7B on the Apple GPU, Q8_0 and Q4_K_M
  • Dropped layers never load
  • A small model drafts, the big one checks: same output, faster

Now

  • Faster Q4 decode
  • Faster first token on long prompts
  • Smart context on the GPU
  • Run the model from its graph

Built, not wired in

  • 2 to 8 bit and ternary compression
  • One model across several devices

Next

  • Expert paging for MoE models
  • Cold KV pages on disk
  • CUDA on NVIDIA
  • More models
  • Orleans cluster

Get started

Fetch. Prepare. Generate.

1 · Fetch the smoke-test model
dotnet run --project src/Synapse.Cli -- model fetch --set smoke
2 · Prepare the model once
dotnet run --project src/Synapse.Cli --configuration Release -- model compile --source artifacts/models/qwen2.5-0.5b-instruct-q8_0/qwen2.5-0.5b-instruct-q8_0.gguf --output artifacts/models/qwen2.5-0.5b-instruct-q8_0/qwen2.5-0.5b-instruct-q8_0.synapse
3 · Generate 8 tokens from .synapse
dotnet run --project src/Synapse.Cli --configuration Release -- generate \
  --model artifacts/models/qwen2.5-0.5b-instruct-q8_0/qwen2.5-0.5b-instruct-q8_0.synapse \
  --tokens 785,6722,315,9625,374 --max-tokens 8

Needs .NET SDK 10.0.401 and Rust 1.98.1. Every command is in AGENTS.md.

Help build the fly-brain runtime.

Synapse is MIT-licensed and built in the open. Star it, try it, break it, send a PR.

Compared against

llama.cpp, Microsoft Foundry Local, MLX via SwiftLM, LLamaSharp, dotLLM (GPL-3.0, run as a separate process only).

Built with

.NET, Rust, ZoneTree, Orleans, TUnit, Qwen2.5, three.js for this page.

Ideas from

FlyWire connectome, Mixture-of-Depths, LayerSkip, MoE offloading, BitNet b1.58, BiLLM, SpQR, PTQ1.61, QSpec.

Built by Managed Code

Synapse is built by Managed Code - we build custom AI agents and AI automation on .NET that run in production, with tests, guardrails, and a human in the loop.