vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- vllm delivers high-throughput inference for large language models while keeping memory usage low.
- It supports efficient model serving, enabling faster response times in production environments.
- The engine is open-source and integrates easily with existing ML pipelines.
A high-throughput and memory-efficient inference and serving engine for LLMs
- Price: Free — Open source
- Category: Research
- This week: +201 GitHub stars this week
- Source on GitHub (93,149 stars)
- First seen by Liz:
Visit vllm
More in Research
- ponytail — Makes your AI agent think like the laziest senior dev in the room. The best code is the…
- ECC — The agent harness performance optimization system. Skills, instincts, memory, security,…
- firecrawl — Supercharge your AI agents with data from the web and beyond. Building the library for…
- caveman — 🪨 why use many token when few token do trick. Viral skill + proxy for coding agents that…
- MoneyPrinterTurbo — 利用 AI 大模型和自动化工作流，根据主题或关键词一键生成高清短视频。Generate HD short videos from a topic or keyword with…
- graphify — Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable…
- mcp-for-blender — Community plugin to control Blender 3D with any LLM of your choice. Not affiliated with…
- opencodex — Universal provider proxy for OpenAI Codex & Claude Code — use any LLM (Claude, Gemini,…
Opening Liz…
