Skip to content

feat(inference): auto-detect local providers in obol sell inference#308

Merged
OisinKyne merged 3 commits intomainfrom
feat/first-run-inference-detect
Apr 6, 2026
Merged

feat(inference): auto-detect local providers in obol sell inference#308
OisinKyne merged 3 commits intomainfrom
feat/first-run-inference-detect

Conversation

@bussyjd
Copy link
Copy Markdown
Collaborator

@bussyjd bussyjd commented Mar 31, 2026

What changed (pivot from review feedback)

Removed the first-run wizard that pushed users to sell inference on install. Instead, the auto-detection now lives inside obol sell inference — it runs only when the user explicitly chooses to sell and doesn't specify --model.

How it works

When obol sell inference <name> is run without --model in an interactive terminal:

  1. Scans common local ports (8080 llama-server, 11434 Ollama, 8000 vLLM, 4000 LiteLLM)
  2. Fingerprints server type via /api/tags, /health, /v1/models
  3. Enumerates available models
  4. Presents a selection:
Detected local inference servers:

  http://127.0.0.1:11434 (ollama) [healthy]
    - qwen3.5:9b (by library)
    - llama3.2:3b (by library)

  [1] qwen3.5:9b — http://127.0.0.1:11434 (ollama)
  [2] llama3.2:3b — http://127.0.0.1:11434 (ollama)

Select [1]: 

If --model is provided, detection is skipped entirely.

Files

File Lines Change
internal/inference/detect.go ~260 Keep: endpoint probing + FormatEndpointDisplay moved here from wizard
internal/inference/detect_test.go ~120 Keep: 6 tests unchanged
cmd/obol/sell.go +40 Wire detection into sellInferenceCommand action
internal/inference/wizard.go deleted First-run wizard removed
internal/inference/wizard_test.go deleted Wizard tests removed

What was removed and why

The first-run wizard assumed users want to sell local inference on first install ("earn first, spend later"). The review feedback was clear: there's no PMF for that. The detection infrastructure is valuable — it just belongs in obol sell inference, not in a first-run flow.

Test plan

  • go build ./cmd/obol/ compiles
  • go test ./internal/inference/ — 6 detection tests pass
  • Manual: run obol sell inference test without --model with Ollama running
  • Manual: run obol sell inference test --model qwen3.5:9b — skips detection

bussyjd added 2 commits March 31, 2026 22:04
Adds automatic discovery of local inference servers (llama-server, ollama,
vllm, litellm) during first-run setup. Probes common ports, hits /v1/models
to enumerate available models, and guides the user through selection.

If no local inference found, shows network preview and suggests obol sell
as the onramp (earn first, spend later — no wallet needed).

Refs: #307
…nfig, security limits, test coverage

- Make ScanLocalEndpoints concurrent (parallel port probing)
- Check context cancellation in scan loop
- Add io.LimitReader to prevent oversized responses
- Remove unused serverType parameter from fetchModels
- Share http.Client across probe calls
- BUG FIX: wizard now persists InferenceConfig to inference.json
- Handle EOF on stdin in wizard
- Add ErrUserDeclined sentinel error
- Add happy-path probe test with httptest server
- Add malformed JSON, context cancellation, empty display tests
- Use dynamic port allocation in tests

Refs: #307
@OisinKyne
Copy link
Copy Markdown
Contributor

I'm unconvinced users want to be pushed to sell their (mostly insufficient) local inference on first install. I don't think there's pmf for it. If we want to make obol sell inference more able to auto detect different types of providers, that i could be down for, but i don't think this pr as is makes sense.

Remove the first-run inference wizard (wizard.go) — users don't want
to be pushed to sell local inference on first install.

Instead, wire the endpoint detection into `obol sell inference`: when
--model is not specified in an interactive terminal, scan local ports
for running inference servers (Ollama, llama-server, vLLM, LiteLLM),
display discovered models, and let the user pick.

- Delete wizard.go and wizard_test.go (first-run flow, "earn first" messaging)
- Move FormatEndpointDisplay() from wizard.go to detect.go (reusable)
- Add auto-detection block in sellInferenceCommand action
- Keep detect.go + detect_test.go unchanged (good infrastructure)
@bussyjd bussyjd changed the title feat(inference): first-run wizard + local endpoint auto-detection feat(inference): auto-detect local providers in obol sell inference Apr 6, 2026
@bussyjd
Copy link
Copy Markdown
Collaborator Author

bussyjd commented Apr 6, 2026

agreed — pivoted. removed the first-run wizard entirely (wizard.go deleted, -400 lines). the "earn first spend later" flow is gone.

kept the detection infra (detect.go — port scanning, server fingerprinting, model enumeration) and wired it into obol sell inference instead. now when you run obol sell inference myapi without --model, it scans local ports, shows what's running, and lets you pick. if --model is provided it skips detection entirely.

tldr: detection is opt-in at sell time, not pushed on first install.

@OisinKyne OisinKyne merged commit 0bee968 into main Apr 6, 2026
6 checks passed
@OisinKyne OisinKyne deleted the feat/first-run-inference-detect branch April 6, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants