The search worked. It took around thirteen seconds, and by the time the results landed the user had usually gone. The reflex was to add infrastructure. Scaling made the economics worse and left the latency roughly where it was, because more capacity multiplies a cost that every search pays for itself, and does nothing about how long any one search takes.
Servers let you run more searches at once. They do not make any one search faster.
Two different numbers hide inside the word fast. Throughput is how many searches a system can handle at the same time. Latency is how long one search takes from the moment someone presses the button. Adding servers raises throughput. It does almost nothing for latency, because a single search still runs along one path and still does all of its own work in the same order, however many other machines are standing by.
Inside a flight search, that work is mostly waiting, and it comes in two kinds. The first is the database: looking up airports, routes, transfer rules and cached prices. A common pattern is a loop that fetches one row at a time, so a search touching many hubs makes many small trips to the database, each cheap on its own and expensive in total. The second is the providers. If the code asks them one after another, the search waits for the sum of their response times. If it asks them all at once, it waits for roughly the slowest. With several providers, that difference can outweigh everything else in the search.
Adding servers can then make things worse. Every new server opens its own connections to the same database and sends its own calls to the same providers. The database serves more simultaneous trips, and each one queues a little longer. Providers enforce their rate limits sooner, so calls are refused or retried. Because each search still does the same wasteful work, only now spread across more machines, the cost per search rises. The bottleneck was never the number of servers. It was the work each search does, and the shared resources every server ends up competing for.
The first job was to find out where a single search actually spent its time, before changing anything. The answer sat in three places: the number of database queries, provider calls made more than once, and a provider fan-out that ran one call after another. Cutting the work inside one search is the only move that improves latency and cost at the same time. Adding capacity improves neither.
Slow systems are usually slow per request, and scaling hides that until the bill arrives.
When a system slows down under load, the natural assumption is that it needs more room. Sometimes it does. More often it is doing too much for every request, and load only makes that visible. The culprits are much the same everywhere: queries issued one at a time inside a loop, the same data fetched several times in one request by different parts of the code, external calls made in sequence, and nothing measuring what a single request costs.
Scaling hides these problems for a while. Throughput improves, and dashboards that show average load look healthier. Latency for any single user barely moves, the shared database and external providers get busier, and the bill grows in step with the waste. By the time someone profiles a single request, the system has often been scaled more than once around a problem that one profile would have found.
The reflex, and the fix.
Scale the infrastructure
It is the standard response to slowness under load. It needs no diagnosis, and it can be done in an afternoon.
More servers multiply a cost that is paid per search. The search was not waiting for capacity. It was doing far too much work, and every new server paid for that work again.
Profile one search and remove the work
A diagnosis phase before any change, and query-level work that is slower to do than provisioning.
The same hardware running the search at 2.8 seconds median instead of 12.8, at roughly a third of the infrastructure cost per search.
Profile before provisioning. Capacity cannot fix a cost that every search incurs on its own.
Profile one search end to end
Before any change, find out where the time goes. A single search was making 149 database queries, which is the kind of number nobody sees until somebody looks.
Remove the repeated reads
Batching and caching brought the count down to 38. Most of the queries removed were the same lookups repeated inside a loop. In the code each one looks harmless. In a profile they stand out at once.
Run the fan-out concurrently, with a bound
The provider calls ran one after another. Running them at the same time, under a ceiling, turned a search that waited for the sum of every provider's response into one that waits for something close to the slowest.
Count cost per search
Infrastructure cost per search was tracked next to latency, so every change could be shown to improve both, with neither traded against the other.
What each choice cost, and what it bought.
AProfiling before provisioning
The first phase changed nothing. It measured one search from end to end while the product stayed slow, and that is uncomfortable when everyone can see the problem and an obvious lever is sitting there. What it bought was the knowledge that the time lived in 149 queries, repeated provider calls and a serial fan-out, which told the work exactly where to go. Provisioning first would have been quicker to start, and as the earlier scaling had already shown, it would have raised the bill while leaving the latency roughly where it was.
BCutting the queries at their source
Repeated lookups inside loops were replaced by batched reads that fetch a whole set at once, and stable data was cached. This is query-level work in code that already ran, which is slower and more careful than adding database capacity. It took the count from 149 queries per search to 38. A bigger database would have served those 149 queries a little faster each, and every search would still have made all of them.
CRunning the provider calls concurrently, under a ceiling
Asking providers at the same time changes what a search waits for, from the sum of their response times to something close to the slowest one. The ceiling is the cost: the widest searches still queue some calls behind others. It is also what keeps the burst inside what providers will accept. Fully serial calls make latency grow with every provider, and unbounded concurrency trades that for rate-limit refusals and retries.
DMeasuring cost per search beside latency
Infrastructure cost was attributed to individual searches and tracked next to their timings. That instrumentation is extra work with no visible feature at the end of it. It is also the evidence. It showed cost falling from 9.5 cents to 2.9 cents per search alongside the median falling from 12.8 to 2.8 seconds, which proves the improvement came from removing work. A monthly bill could never have shown that, because traffic changes it as much as engineering does.
A single headline number hides where a system fails. This work was scored on the dimensions that actually decide whether it holds in production, measured on real, held-out cases rather than the demo path.
Latency, query-count and infrastructure-cost figures are measured before and after, on the delivered system.
A larger, more expensive cluster running the same wasteful search, where the median improves slightly and the bill does not.
A search at 2.8 seconds median and 7.5 seconds at the ninety-fifth percentile, down from 12.8 and 29.6, making 38 database queries instead of 149, at 2.9 cents of infrastructure per search instead of 9.5.
What it owns, and what it hands to a person.
The profile was taken on the actual query mix. A very different shape of search, with far more hubs or much longer date ranges, would need its own profile. The speed came from removing work, so it lasts exactly as long as the work stays removed, and a later change that puts a query back inside a loop will undo it quietly. Time spent inside a provider's own response is outside anything this work could change.
When a system is slow at scale and gets worse when scaled, the cause is almost always work per request and seldom capacity. Profile one request, cut the queries and the redundant calls, and only then scale what is left.
Throughput and latency are different numbers. More servers raise how many requests run at once and do almost nothing for how long one request takes.
Adding servers can make a system slower when they all compete for the same database and the same external rate limits.
Profile one request before changing anything. Counts like queries per request reveal problems that code review and averages hide.
External calls made in sequence cost the sum of their response times. Made concurrently under a ceiling, they cost roughly the slowest one.
Track cost per request next to latency. When both fall together, the improvement came from removing work, and it will hold as long as the work stays removed.