Alternatives to vllm

Tools that do the most similar job to vllm, closest first.

• airllm — AirLLM 70B inference with single 4GB GPU
• Jinfer — AI inference engine for the JVM. AI in a jar
• unsloth — Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3,…
• haystack — Open-source AI orchestration framework for building context-engineered, production-ready LLM applications.…

Opening Liz…