Janas is a pure C inference runtime for running language models larger than your RAM, on CPU or GPU. The authors report 37 tokens per second from Qwen3-Next-80B, a 48 GB model, on a 32 GB laptop against 2.8 for llama.cpp, and faster results on all 13 models they compared at Q4_K_M.
Be the first to rate Janas. Your feedback helps other users decide.
Stream weights on a machine with limited memory
Measure throughput against llama.cpp on your own hardware
Host a model without a GPU cluster
Similar tools in the same category — compare side-by-side
Cactus Hybrid post-trains small on-device models to know when they are wrong. Every answer carries a...
LittleBit is the official implementation of sub-1-bit large language model compression by latent fac...
Agentic CUDA Kernel Optimizer is a LangGraph workflow wrapped around a C++ CUDA harness that optimiz...
Guides and comparisons about code tools
Eight local inference projects that fit big models into small memory: a 26B model in 2 GB, MoE weight streaming, a console as a server and sub-1-bit compression
Nine Show HN launches beneath the model: filesystems, snapshots, gateways and deploy tools that decide what an agent sees and what survives.
Nine launches that check agent work: per-call access policy, MCP server vetting, runtime verification, readable diffs and a hard stop at 95%.
Nine Show HN launches sitting under the agent: a model gateway, vectorless retrieval, on-device inference, an MCP browser layer and connectors for real systems.
Eight Show HN projects for trusting long agent runs: work context capture, distilled memory, folder fences, dollar budgets, plan review and tool governance.