@rsudqprsudqpbnhu
We are thrilled to announce the milestone release of SGLang Runtime v0.2, featuring significant inference optimizations after months of hard work.
It achieves up to 2.1x higher throughput compared to TRT-LLM and up to 3.8x higher throughput compared to vLLM. It consistently https://t.co/w1wglPpxs8
0 reply
0 recast
0 reaction