feat(inference): auto-detect local providers in obol sell inference#308
feat(inference): auto-detect local providers in obol sell inference#308
Conversation
Adds automatic discovery of local inference servers (llama-server, ollama, vllm, litellm) during first-run setup. Probes common ports, hits /v1/models to enumerate available models, and guides the user through selection. If no local inference found, shows network preview and suggests obol sell as the onramp (earn first, spend later — no wallet needed). Refs: #307
…nfig, security limits, test coverage - Make ScanLocalEndpoints concurrent (parallel port probing) - Check context cancellation in scan loop - Add io.LimitReader to prevent oversized responses - Remove unused serverType parameter from fetchModels - Share http.Client across probe calls - BUG FIX: wizard now persists InferenceConfig to inference.json - Handle EOF on stdin in wizard - Add ErrUserDeclined sentinel error - Add happy-path probe test with httptest server - Add malformed JSON, context cancellation, empty display tests - Use dynamic port allocation in tests Refs: #307
|
I'm unconvinced users want to be pushed to sell their (mostly insufficient) local inference on first install. I don't think there's pmf for it. If we want to make |
Remove the first-run inference wizard (wizard.go) — users don't want to be pushed to sell local inference on first install. Instead, wire the endpoint detection into `obol sell inference`: when --model is not specified in an interactive terminal, scan local ports for running inference servers (Ollama, llama-server, vLLM, LiteLLM), display discovered models, and let the user pick. - Delete wizard.go and wizard_test.go (first-run flow, "earn first" messaging) - Move FormatEndpointDisplay() from wizard.go to detect.go (reusable) - Add auto-detection block in sellInferenceCommand action - Keep detect.go + detect_test.go unchanged (good infrastructure)
|
agreed — pivoted. removed the first-run wizard entirely (wizard.go deleted, -400 lines). the "earn first spend later" flow is gone. kept the detection infra (detect.go — port scanning, server fingerprinting, model enumeration) and wired it into tldr: detection is opt-in at sell time, not pushed on first install. |
What changed (pivot from review feedback)
Removed the first-run wizard that pushed users to sell inference on install. Instead, the auto-detection now lives inside
obol sell inference— it runs only when the user explicitly chooses to sell and doesn't specify--model.How it works
When
obol sell inference <name>is run without--modelin an interactive terminal:/api/tags,/health,/v1/modelsIf
--modelis provided, detection is skipped entirely.Files
internal/inference/detect.goFormatEndpointDisplaymoved here from wizardinternal/inference/detect_test.gocmd/obol/sell.gosellInferenceCommandactioninternal/inference/wizard.gointernal/inference/wizard_test.goWhat was removed and why
The first-run wizard assumed users want to sell local inference on first install ("earn first, spend later"). The review feedback was clear: there's no PMF for that. The detection infrastructure is valuable — it just belongs in
obol sell inference, not in a first-run flow.Test plan
go build ./cmd/obol/compilesgo test ./internal/inference/— 6 detection tests passobol sell inference testwithout--modelwith Ollama runningobol sell inference test --model qwen3.5:9b— skips detection