Jianwei Yu 3817f74d46 feat: nginx-based data parallel for optimal ASR throughput ...

When --dp N is specified (N > 1), the launcher now starts N independent
vLLM processes behind an nginx reverse proxy instead of using vLLM's
built-in DP coordinator. This avoids the single-process HTTP bottleneck
when handling large base64 audio payloads, achieving near-linear scaling
(7.2x with 8 GPUs at 4096 concurrent requests).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-03-27 07:43:32 +00:00

..

gradio_asr_demo_api_video.py

Add Gradio ASR demo with video support and demo audio/video files

2026-03-22 06:11:51 +00:00

start_server.py

feat: nginx-based data parallel for optimal ASR throughput

2026-03-27 07:43:32 +00:00