Skip to content

feat(inference): support plaintext HTTP in inference interception #38

@pimlock

Description

@pimlock

Context

Follow-up from debugging sandbox sandbox-55c06ef3-5e99-4942-a342-37abab656f52. Related to #37.

When a sandboxed app makes a plaintext HTTP request (e.g., http://test:8000/v1/chat/completions), the CONNECT tunnel is established correctly and OPA returns InspectForInference, but handle_inference_interception() unconditionally TLS-terminates the connection. The client sends POST /v1/... (byte 0x50), rustls expects a TLS ClientHello (byte 0x16), and fails with:

WARN navigator_sandbox::proxy: Proxy connection error error=received corrupt message of type InvalidContentType

Proposed Solution

Peek the first byte after sending "200 Connection Established" to detect TLS vs plaintext HTTP, then branch accordingly. This follows the existing TLS-vs-plaintext pattern in the L7 relay path (proxy.rs:399-493).

Changes

1. crates/navigator-sandbox/src/proxy.rs

Add is_tls_record_byte helper (~line 650):

  • TLS records start with content type 0x14..=0x18 (Handshake=0x16 is most common)
  • Plaintext HTTP starts with an ASCII letter (P, G, D, H, etc.)

Extract inference_read_loop from handle_inference_interception (pure refactor):

  • Move lines 686-730 (the buffer read/parse/route loop) into a new generic function
  • Signature: stream: &mut (impl AsyncRead + AsyncWrite + Unpin)
  • route_inference_request is already generic over AsyncWrite + Unpin

Modify handle_inference_interception:

  • After verifying inference_ctx, peek 1 byte from the TcpStream
  • If is_tls_record_byte: require tls_state, TLS-terminate, run inference_read_loop on TLS stream
  • If is_ascii_alphabetic(): log at debug, run inference_read_loop directly on raw TcpStream
  • Otherwise: return error with the unexpected byte value

2. crates/navigator-sandbox/src/opa.rs

Update doc comment on InspectForInference variant (~line 28) — replace "TLS-terminate" with "inspect the tunnel (TLS-terminating if the client speaks TLS)".

Tests

  • Unit test: is_tls_record_byte classification — TLS content types vs HTTP method bytes vs edge cases
  • E2E test: extend e2e/python/test_inference_routing.py with a variant using http:// instead of https://

Verification

  1. cargo build -p navigator-sandbox
  2. cargo test -p navigator-sandbox (including new unit test)
  3. mise run pre-commit
  4. Manual: run sandbox with inference routing, make HTTP request to inference endpoint, verify success

Originally by @pimlock on 2026-02-22T23:36:19.148-08:00

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:inferenceInference routing and configuration work

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions