Jianwei Yu 3817f74d46 feat: nginx-based data parallel for optimal ASR throughput ...

When --dp N is specified (N > 1), the launcher now starts N independent
vLLM processes behind an nginx reverse proxy instead of using vLLM's
built-in DP coordinator. This avoids the single-process HTTP bottleneck
when handling large base64 audio payloads, achieving near-linear scaling
(7.2x with 8 GPUs at 4096 concurrent requests).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-03-27 07:43:32 +00:00

..

vibevoice-asr.md

update README

2026-01-27 21:06:31 +08:00

vibevoice-realtime-0.5b.md

streaming use transformers==4.51.3

2026-02-03 00:30:52 -08:00

vibevoice-tts.md

restructure README

2026-01-22 00:37:22 -08:00

vibevoice-vllm-asr.md

feat: nginx-based data parallel for optimal ASR throughput

2026-03-27 07:43:32 +00:00