An operating system for LLM inference
One binary. No dependencies. Drop it in and run. Up to 4.62x faster prefill than llama.cpp and 5.53x faster batched decoding than vLLM. jitllm detects your hardware and optimizes kernels and model placement across CPUs and GPUs. Run open-weight flagships, even larger than memory. OpenAI- and Anthropic-compatible APIs, desktop and terminal apps. Linux, macOS and Windows.
jitllm is an operating system designed for large language model (LLM) inference, offering a single binary with no dependencies for easy deployment. It provides significant performance improvements over existing solutions, optimizing for both CPU and GPU hardware while supporting OpenAI and Anthropic APIs across multiple operating systems.
A source that found nothing is a measurement. A source that has not run is a gap. Neither means the launch lacks the thing.