Give Your AI Server A Checkup With Vllm-Doctor
Vllm-doctor is a new command-line tool that diagnoses performance bottlenecks in vLLM inference servers by reading live metrics. It turns raw data into clear explanations of what is wrong, why it may be happening, and which server settings to adjust. The tool operates as a quick diagnostic snapshot, not a full monitoring dashboard.
Developer aminalaee created vllm-doctor to help self-hosted AI users spot problems like queue pressure, KV cache overload, and slow token generation. The open-source project runs directly against a server’s /metrics endpoint or a Prometheus instance without requiring extra setup. It fills a gap for users who need actionable advice without learning to interpret complex metric graphs.
Fast server-side diagnostics
Key Features
- Built-in rules detect queue pressure and bottlenecks.
- Save and review diagnostic history locally.
- Watch mode logs only when health changes.
- Supports Prometheus or direct metrics scrape.
- Rich text tables or JSON for automation.
- Configurable thresholds via TOML file.
Anyone running a vLLM inference server can use vllm-doctor to quickly find performance issues. Instead of learning to interpret raw metrics, they get clear recommendations for configuration changes.
Project status and feedback
vllm-doctor is still in early development, and the creator welcomes suggestions for new diagnostic rules. Unlike a dashboard, it does not replace monitoring tools but delivers quick snapshot diagnostics for a single server or Prometheus target. It pairs well with GuideLLM, which generates workloads, while vllm-doctor explains the resulting server-side bottlenecks.
"vLLM Doctor reads vLLM server metrics and turns them into diagnostic findings: what looks unhealthy, why it may be happening, and which vLLM settings are worth checking first." Source: GitHub
Project Links
