docs: Document GPU priority and worker distribution

[promptless]

Open this suggestion in Promptless to view citations and reasoning process

Expands the GPU configuration section with a dedicated subsection explaining how Runpod distributes workers across multiple GPU types. Documents that with 5+ active workers, Runpod distributes workers across GPU priorities (most on primary type), and with fewer than 5 workers, all use the highest-priority available GPU. Also adds a cross-reference from the optimization guide.

Trigger Events

Internal discussion about GPU Failover feature documentation and UI changes, specifically clarifying the minimum 5-worker requirement for the failover logic to work properly before the beta release.

Promptless Research (6 files, 1 Slack conversation)

• Opened Slack conversation
• Viewed .long_term_context/product_knowledge/product_overview.md
• Viewed .long_term_context/doc_workflow/client_instructions.md
• Viewed serverless/endpoints/endpoint-configurations.mdx
• Viewed serverless/endpoints/overview.mdx
• Viewed .long_term_context/style/client_style_guide.md
• Viewed flash/configuration/gpu-types.mdx

Agent Response

I've added documentation clarifying the GPU failover worker requirement based on the Slack thread discussion.

Summary

Change made: Added a note to the Endpoint settings page explaining that GPU priority distribution requires at least 5 max workers.

Why: The Slack thread identified that users were confused when GPU failover appeared not to work. The underlying issue is that the worker distribution uses an 80/15/5 ratio across GPU priorities, and with fewer than 5 workers, rounding causes secondary and tertiary GPU types to receive zero workers—effectively disabling failover.

What the note says:

When you select multiple GPU types, Runpod distributes workers across them using a priority ratio. You need at least 5 max workers for this distribution to work—with fewer, rounding causes secondary and tertiary GPU types to receive zero workers, so your endpoint only uses the primary type.

The suggestion is ready for review: https://app.gopromptless.ai/suggestions/2ed787a3-b919-4045-a598-2b522d2954a4

Internal discussion about updating Serverless documentation with GPU priority and worker distribution optimization information.

Promptless Research (5 files)

• Viewed .long_term_context/doc_workflow/client_instructions.md
• Viewed .long_term_context/style/client_style_guide.md
• Viewed .long_term_context/product_knowledge/product_overview.md
• Viewed serverless/endpoints/endpoint-configurations.mdx
• Viewed serverless/development/optimization.mdx

Agent Response

I've updated the Serverless documentation with the GPU priority and worker distribution information. Here's a summary of the changes:

Updated suggestion: Document GPU priority and worker distribution

Changes made:

1. Endpoint settings page (serverless/endpoints/endpoint-configurations.mdx):

   • Added a new "GPU priority and worker distribution" subsection under GPU configuration
   • Explains how to specify up to three GPU types in priority order
   • Documents worker distribution behavior with 5+ active workers (most run on primary, fewer on secondary/tertiary)
   • Documents behavior with fewer than 5 workers (all use highest-priority available GPU)
   • Highlights the benefit of reduced throttling during GPU constraints

2. Optimization guide (serverless/development/optimization.mdx):

   • Added a cross-reference from the "Select multiple GPU types" section linking to the detailed explanation

The content was placed in the endpoint settings reference page rather than solely in the optimization guide since it's a configuration-related feature, with a brief mention and link from the optimization guide for discoverability.

---

Tip: Send Promptless a meeting transcript in Slack to generate doc updates 📝

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
runpod-docs	🟢 Ready	View Preview	Apr 1, 2026, 1:36 PM

[promptless]

Just a reminder: If you'd like me to act on any feedback you have via Github comments, just type @Promptless in your suggestion and I'll get right on it! (I won't show up in the user dropdown, but I'll process any request that has @Promptless in the comment body.)