Enki LLM is listed as Open Source. Check https://github.com/enkiruntime/enki_llm for the current plan limits, as vendors change pricing frequently.

Enki LLM

Q: What is Enki LLM?

Enki LLM is a code AI tool listed on Use AI Tools. Enki LLM is a pure Rust Transformer inference engine of about 2,000 lines that runs Llama-3.2 1B at roughly 40 tokens per second on an Nvidia Tesla T4, with live brain migration between GPU VRAM and CPU RAM mid generation and no CUDA dependencies.

Q: What is Enki LLM best for?

Enki LLM is commonly used for Running a small model in pure Rust, Migrating a running generation between GPU and CPU, Studying a small, readable inference engine. It supports English.

Code⚡ Advanced✅ Global access

Running a small model in pure RustMigrating a running generation between GPU and CPUStudying a small, readable inference engine

Open SourceEnglish

Enki LLM is a pure Rust Transformer inference engine of about 2,000 lines that runs Llama-3.2 1B at roughly 40 tokens per second on an Nvidia Tesla T4, with live brain migration between GPU VRAM and CPU RAM mid generation and no CUDA dependencies.

Visit Website Compare

Community Rating

Be the first to rate Enki LLM. Your feedback helps other users decide.

🎯 Real-World Use Cases

Infer without CUDA

Unquantized F16 weights with FP32 accumulators run in pure Rust, with no CUDA or C++ wrapper.

Migrate mid-sentence

Live brain migration hot-swaps active execution and mirrors the KV-cache between GPU VRAM and CPU RAM without losing context.

Run one binary everywhere

The same binary runs on discrete Nvidia GPUs, integrated Intel graphics and multi-core CPUs without changing a line of code.

⚖️ Honest Review

✅ Pros

+Around 2,000 lines with the safetensors parser, tokenizer, RoPE, RMSNorm and sampling built from scratch
+One-click Google Colab demo on a free T4 GPU
+Host fallback executes the same kernels across CPU cores with Rayon

⚠️ Cons

−Demonstrated on Llama-3.2 1B, so it is a small-model engine rather than a serving stack
−Depends on the separate Enki runtime and Vulkan 1.3

📚

Guides and comparisons about code tools

CodeOct 26, 2026