vllm-project/vllm

vllm-project/vllm

Releases167
Frequency6 days 23 hours
Last Release
Stars83.5K
A high-throughput and memory-efficient inference and serving engine for LLMs

Linked projects

A high-throughput and memory-efficient inference and serving engine for LLMs