vllm-project/vllm

vllm-project/vllm

Releases175
Frequency6 days 19 hours
Last Release
Stars86.2K
A high-throughput and memory-efficient inference and serving engine for LLMs

Linked projects

A high-throughput and memory-efficient inference and serving engine for LLMs