> For the complete documentation index, see [llms.txt](https://docs.podtorrent.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.podtorrent.com/concepts/routing.md).

# Routing and queueing

For each request the gateway picks one node. Here's how.

## Which nodes qualify

A node can take the request only if all of these hold:

* It serves the requested model.
* It accepts the request's tier.
* Its output price for that tier is within the client's ceiling, if one was set.
* It has been heard from in the last 90 seconds.
* It has a free slot. Batch requests can't use a model's `interactive_reserve` slots.

## How the winner is chosen

Among the nodes that qualify, in this order:

1. **The client's operator's own nodes** (only for [operators' own clients](/for-operators/own-clients.md)).
2. **A node with the request's opening cached** (see [cached prompts](/using-the-api/cached-prompts.md)).
3. **Free slots.**
   * Interactive: the node with the **most** free slots. Less busy means faster answers.
   * Batch: the node with the **fewest** free slots that still has one. This fills up batches that are already running, which is cheaper per token on GPU hardware.
4. **The lower price.**

## Counting slots

The gateway counts the requests it has sent to each node, and also listens to what the node reports. A node that says it's busier than the gateway thinks (maybe it's doing other work) is treated as busier. The network never sends a node more requests than its slots.

## When everything is full

If some node qualifies but none has a free slot, the request waits:

| Tier        | Waits up to |
| ----------- | ----------- |
| interactive | 10 seconds  |
| batch       | 2 minutes   |

If a slot frees up, the request goes there. If not, the client gets a `503` `capacity_error`. If no node serves the model at all, the `503` comes right away.

## When a node fails

If the chosen node fails before sending any data, the gateway tries one other node. If that fails too, the client gets a `503` `capacity_error`.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.podtorrent.com/concepts/routing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.