Skip to content

feat(server): add configurable worker count to prevent single-worker blocking#470

Merged
MaojiaSheng merged 1 commit intovolcengine:mainfrom
r266-tech:fix/multi-worker-server
Mar 7, 2026
Merged

feat(server): add configurable worker count to prevent single-worker blocking#470
MaojiaSheng merged 1 commit intovolcengine:mainfrom
r266-tech:fix/multi-worker-server

Conversation

@r266-tech
Copy link
Contributor

Summary

Adds a --workers CLI flag and server.workers config option to allow running the OpenViking HTTP server with multiple uvicorn worker processes.

Problem

As reported in #464, OpenViking runs with a single uvicorn worker by default. A single slow or blocking request (e.g. a heavy commit operation) can stall the entire server, causing even lightweight endpoints like /health to become unresponsive. This is a serious availability issue for production deployments.

Solution

  • Add workers: int = 1 field to ServerConfig
  • Add --workers CLI argument to openviking-server
  • Support server.workers in ov.conf config file
  • When workers > 1, use uvicorn's factory mode with an import string so each worker process can independently initialize the application
  • Default remains 1 to preserve existing single-worker behavior

Configuration

# CLI
openviking-server --workers 4

# ov.conf
{
  "server": {
    "workers": 4
  }
}

Changes

  • openviking/server/config.py: Add workers field to ServerConfig and load from config
  • openviking/server/bootstrap.py: Add --workers CLI arg, multi-worker uvicorn launch with factory mode

Notes

  • Minimal, backward-compatible change — default behavior (single worker) is unchanged
  • Multi-worker mode uses uvicorn's factory=True with the app import string, which is the standard approach for multi-process uvicorn
  • For more advanced concurrency needs (e.g. wrapping blocking I/O in run_in_executor), that can be addressed in follow-up PRs

Closes #464

…blocking

Add --workers CLI flag and server.workers config option to allow running
uvicorn with multiple worker processes. This prevents a single slow or
blocking request from stalling the entire HTTP server, including
lightweight endpoints like /health.

When workers > 1, the server uses uvicorn's factory mode with an import
string so each worker process can independently initialize the
application.

Configuration:
  - CLI: openviking-server --workers 4
  - ov.conf: { "server": { "workers": 4 } }
  - Default: 1 (preserves existing behavior)

Closes #464
@MaojiaSheng MaojiaSheng merged commit 5b9b533 into volcengine:main Mar 7, 2026
4 of 6 checks passed
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: OpenViking runs with a single uvicorn worker, allowing one blocking request to stall the entire HTTP server

2 participants