Attention · always runsExpert the router pickedSkipped
Open source · C# on .NET 10 · by Managed Code
Run LLMs like a fly brain.
Synapse runs large language models on your own machine, in pure C#. No Python, no server, no network. And it is designed to fire only the regions each token needs.
LLM engines run every block for every token and keep every weight in memory. A wall of compute, whether the token needs it or not.
02The idea Next
A wave, not a wall.
A fly's brain sends a signal through only the few regions it needs (FlyWire connectome). Synapse does the same with a model: dense attention always runs, the model's own router picks the expert, and a region is skipped only with proof.
03Memory Built · layer drop
Cold regions stay on disk.
A layer a profile drops never loads: its weights stay on disk and it gets no KV cache. Its quality cost is measured before it is dropped.
240 MiBper dropped layer, Qwen2.5-7B
04Scale Built · wiring in next
One model. Several machines.
Regions run on different machines. Only the hidden state travels between them.
3.5 KiBper token for Qwen2.5 0.5B
Latest measurements · GitHub Actions
How does Synapse compare? Pick a model. See the difference.
Green is Synapse. Choose a model, a runner class and an answer length. Each chart names its unit and timing scope; missing measurements stay visible. These short diagnostic runs do not establish a quality or release winner.
Results load from the published JSON. Open the links above to inspect the original runs.
Tests
No run loaded.
Performance
No run loaded.
Synapse vs Microsoft Foundry Local
Foundry Local runs downloaded ONNX models on the runner. This is local inference, not the Azure AI Foundry cloud service. Compare the same model family below; GGUF and ONNX weights, threads and cache policies differ.
Waiting for measured evidence.
Synapse CPU and MLX on Apple GPU
Current hosted measurements use different devices and weights. For Synapse Metal vs llama.cpp Metal on the same local Mac, see the Apple GPU charts below.
Waiting for measured evidence.
Recorded local measurements · September 2026 · Apple M2 Pro
Our models on Apple silicon. Synapse beside the other engines.
These earlier local diagnostics use different hardware from GitHub Actions above. The runnable decoder models shown here are Qwen2.5 0.5B and Qwen2.5 7B. Green is Synapse, blue is llama.cpp, purple is MLX. Check the answer-quality note as well as speed.
MacBook Pro, Apple M2 Pro, 32 GB. Qwen2.5 0.5B Instruct Q8_0, prompt “The capital of France is”, 8-token answer. Median of 5 runs after 3 warm-ups; all four engines wrote the same 8 tokens. Diagnostic runs, not the final 30-run release gate. All runs and raw JSON ↗
Qwen2.5 0.5B · Apple M2 Pro GPU · September 2026
Tokens, memory and speed. From 4k to 32k.
Synapse and llama.cpp use the same Q8_0 file and selected KV precision. MLX uses its own 8-bit weights. Memory is peak physical footprint; it differs from the peak RSS charts above. Correct answers and matching token IDs appear below the charts.
Pick a context and a KV cache
Memory
Synapse534 MiB
llama.cpp535 MiB
MLX13,500 MiB
peak footprint · KV cache 384 MiB
First token
Synapse27.63 s
llama.cpp26.62 s
MLX15.51 s
time to first token · lower is better
Full answer
Synapse27.82 s
llama.cpp26.81 s
MLX17.62 s
Synapse and llama.cpp stop after 19 tokens, MLX writes 128
Writing speed
Synapse92.2
llama.cpp92.9
MLX60.1
tokens per second · higher is better
Same tokens: Synapse wrote the same tokens llama.cpp wrote. Right answer: Synapse 2/2, llama.cpp 2/2, MLX 0/2.
MacBook Pro, Apple M2 Pro 19-core GPU. Qwen2.5 0.5B Q8_0, one prompt per context: repository text with a hidden number, then “state the number and summarize”, up to 128 tokens. Every engine gets the same text and its token IDs are checked. Median of 2 runs after a warm-up, one engine at a time. MLX runs its own 8-bit weights and cache format; Synapse and llama.cpp also map the 644 MiB model file. Perplexity on identical tokens matches llama.cpp within 0.01% at 4k–32k. Run T and raw JSON ↗
A big window, small memory
7×less than llama.cpp
Synapse234 MiB
llama.cpp1,645 MiB
peak footprint · 131k window, 4k prompt, FP16 KV · the KV cache grows with use instead of being reserved
Second question, same document
0.13 sto the first token
Reuse prefix0.13 s
Read it again23.8 s
30,000-token document · Metal, FP16 KV · the same answer either way
One run per row, a fresh process each; diagnostics, not a paired verdict. Runs U and V ↗
Qwen2.5-7B-Instruct-1M · Apple M2 Pro GPU · 29 September 2026
One model. Two engines. Compare the same weight format.
Peak memory
4,241MiB, Synapse Q4_K_M
Synapse4,241 MiB
llama.cpp4,849 MiB
peak RSS, mapped weights included · median of 2 alternating rounds · lower is better
Writing speed
17.8tokens/s, Synapse Q4_K_M
Synapse17.8
llama.cpp28.2
tokens/s · median of 2 alternating rounds · higher is better · Synapse reported decode vs llama.cpp native evaluation: phase definitions differ
Small draft, big check
1.35×faster, same output
+ 0.5B draft23.5
7B alone17.4
tokens per second, Q8_0, chat prompt · the 0.5B drafts 3 tokens, the 7B checks them in one pass, 54% kept
Drop layers, never load them
−475 MiB2 of 28 layers dropped, perplexity −1.1%
All 28 layers7,255 MiB · 12.16
2 dropped6,780 MiB · −1.1%
4 dropped6,326 MiB · +39%
8 dropped5,381 MiB · +137%
peak memory · perplexity on the test text · a dropped layer's weights are never mapped for the GPU and its KV is never allocated · only the 2-layer drop passes the quality gate
Apple M2 Pro 19-core GPU, Qwen2.5-7B-Instruct-1M. The two selected-format engines use the same GGUF file, 3,528-token prompt, 128-token limit and FP16 KV; medians of 2 alternating fresh-process rounds. Exact text parity was checked separately on the pinned 16-token “The capital of France is” prompt; it is not a quality verdict for the long answer. Q4_K_M perplexity is 11.93 vs 12.16 for Q8_0 on 2,048 scored tokens. Diagnostics, not a release verdict. 7B measurements and quality source ↗
Not there yet. We say so.
Prompt processingThe local 0.5B first-token charts show the long-prompt gap. For 7B, Synapse records 16.9–17.3 s to its first token; llama.cpp records 10.8–11.2 s of native prompt evaluation. Those 7B phase definitions differ, so they do not establish a first-token speedup.
Q4 writing speedThe 7B in Q4_K_M uses less peak RSS than llama.cpp in these two rounds. Synapse reports 17.4–18.2 decode tokens/s; llama.cpp reports 27.0–29.3 native-evaluation tokens/s. The phase definitions differ; faster Q4 kernels are next.
Past 32kThe 7B-1M model and head size 128 kernels run now. Windows past 32k have not been measured yet.
Smart context on CPU onlyReading only the relevant KV pages costs +3.4% perplexity while reading 30% of an 8k context. The GPU kernels are next.
CUDA untestedMetal runs. The CUDA backend is written but has not run on an NVIDIA GPU yet; it rejects Q4_K_M files for now.
One familyOnly Qwen2 runs today. More model families and MoE expert paging are next.
Why Synapse
Built for .NET, from the kernels up.
C#
Pure C#, no Python
.NET 10 owns the portable path. Rust takes a kernel only after profiling proves it is worth it.
⌂
Local first
One process, no server, no network. The cluster is an option, never a requirement.
=
Same answer as llama.cpp
Qwen2.5 runs fully in C#. In the benchmark it writes exactly the same tokens as llama.cpp.
≈
Small drafts, exact answer
A 0.5B model drafts tokens and the 7B checks them in one pass. The answer stays exactly the 7B's, 1.35× faster on chat text.
!
Fails loudly
Model math, precision and cache formats are explicit. An unsupported fast path fails instead of guessing.
✓
Verified and cross-platform
Model downloads are verified. Tests and benchmarks run on macOS, Linux and Windows.
Architecture
Decide what, how and where. Then run the wave.
1
Model IR
What can run.
2
Execution IR
How, and how precisely.
3
Deployment plan
Where each region runs.
4
Wave scheduler
Runs only the regions a token needs.
Every worker runs the same runtime
C# kernelsRust kernelsMetalCUDA · untestedWeight residencyHot KV cacheZoneTree state
Local
An app uses Synapse in one process. Works without Orleans and without a network.
Cluster Planned
An Orleans control plane coordinates workers. Workers pass the hidden state directly to each other.
Synapse is built by Managed Code - we build custom AI agents and AI automation on .NET that run in production, with tests, guardrails, and a human in the loop.