The session limit is the number that matters and the one nobody puts in a design document. A baseboard controller is a small embedded computer and it will stop responding rather than queue politely.
| Value | Against | Verdict | |
|---|---|---|---|
| Sessions opened per node | 1 | 4 allowed | within limit |
| Time per node | 1.1 s | one session reused | |
| Work per sweep | 7.4 h | single-threaded | |
| Sweep with 64 workers | 7.0 min | against a 5.0 min interval | does not fit |
| Workers needed | 90 | you are short | add 26 |
Nodes coverable, by worker count and interval
Six metrics per node, 350 ms session setup, 120 ms per metric, with one session reused for every metric. The figure in brackets is the same pool without session reuse, which is what a client that opens a connection per metric achieves. Reuse is worth more than roughly a threefold increase in workers, and it also keeps you inside the session limit.
| Workers | 1 min interval | 5 min interval | 15 min interval | 1 h interval |
|---|---|---|---|---|
| 8 | 448 (170) | 2,242 (851) | 6,728 (2,553) | 26,915 (10,212) |
| 16 | 897 (340) | 4,485 (1,702) | 13,457 (5,106) | 53,831 (20,425) |
| 32 | 1,794 (680) | 8,971 (3,404) | 26,915 (10,212) | 107,663 (40,851) |
| 64 | 3,588 (1,361) | 17,943 (6,808) | 53,831 (20,425) | 215,327 (81,702) |
| 128 | 7,177 (2,723) | 35,887 (13,617) | 107,663 (40,851) | 430,654 (163,404) |
| 256 | 14,355 (5,446) | 71,775 (27,234) | 215,327 (81,702) | 861,308 (326,808) |
Green covers a 25,000 node estate, amber covers 5,000. These are throughput ceilings only: a schedule that fits here can still exceed the per-controller session limit, which is the failure that degrades the fleet rather than merely falling behind.
A worked example
A 25,000 node estate, six metrics per node, on a five-minute poll interval with 64 workers. The client opens a session, reads a metric, closes it, and repeats: six sessions per node per sweep.
On throughput alone it is already failing. Without reuse each node costs 2.8 seconds, so a sweep is nearly 20 hours of work and 64 workers deliver it in about 18 minutes against a five-minute interval. The polls queue, the fleet is never fully covered, and the platform reports lag rather than failure.
The worse problem is the one that does not appear in any throughput number. Six sessions per node against a limit of four means the controllers start refusing connections, and the platform reads a refusal as an unhealthy node, and unhealthy nodes are polled more aggressively. The monitoring degrades what it monitors, and the harder you resource it the faster that happens.
Reusing one session for all six metrics takes the per-node cost to 1.07 seconds and the sweep to about 7 minutes on the same 64 workers. That is still outside the five-minute interval, so the pool needs to reach 89 workers to close it. But it also puts one session on each controller instead of six, and that is the change worth making first: the throughput shortfall is lag, while the session overrun is the platform degrading the fleet it is there to watch.
The arithmetic, so you can check it
With session reuse a node costs setup + metrics x per-metric response. Without it, every metric pays setup again: metrics x (setup + per-metric). A sweep is nodes x per-node / workers, and it is feasible when that fits inside the interval.
Detection latency for a silent node is interval x consecutive misses required. The session check is separate and it is the one that matters: sessions opened per node against what the controller allows, which a global worker count cannot express at all.
This is a throughput and concurrency model, not a protocol one. Real estates are heterogeneous in ways that change the arithmetic per node: iDRAC 8 and 9 differ, iLO 4 and 5 differ, optional schema fields drop inconsistently, and error behaviour is not uniform even across firmware revisions of one vendor. Response times vary by an order of magnitude across a fleet purchased as uniform. Measure a sample of each variant you actually have, and treat a single per-metric figure as the optimistic case.
The three failures that only exist at scale are in fleet scale changes the failure mode, and the retransmission loop this resembles in bounded the wrong thing.
Questions
Typically four to eight, and it varies by vendor and by firmware revision on the same vendor. It is a small embedded computer, so it stops responding rather than queueing politely, and a monitoring platform that exceeds the limit degrades the thing it is monitoring. Check the controllers you actually have rather than the specification.
Because the constraint is per-target, not global. More workers means more simultaneous sessions landing on individual controllers, and once a controller stops responding the platform records the node as unhealthy and polls it harder. That feedback loop has the same shape as the retransmission collapse that took the internet down in 1986, and capacity is not what fixes it.
Against the per-node session limit rather than a global concurrency setting, which cannot express the constraint. One session per node collecting every metric, with the scheduler aware of which nodes have an open session, is the shape that works. Retrofitting that onto a worker pool with a concurrency number means rewriting the scheduler, which is why it is the item worth getting right first.
The poll interval multiplied by the number of consecutive misses you require before declaring it. One missed poll is noise, since nodes are legitimately offline, in maintenance or slow. The rule is what turns absence into a signal, and without one nothing distinguishes a dark node from one that has not been polled yet.
Not the signals that matter. ECC error counts, thermal throttling state and XID errors are not exposed over the out-of-band path at all, so no polling schedule reaches them. They require in-band agents on the host, which is a decision about your deployment model rather than a feature to add later.