Tagged cost
3 pieces
- · 10 min read
Output Tokens Are The Bill: Why Trimming Your Prompt Did Not Help
For most model-backed features the output side dominates the cost, because output tokens are priced at a multiple of input tokens and because caching makes repeated input cheaper still. Teams optimise input context anyway, since that is the part they can see and control. Output length is also what drives latency, and the two compound through retries. The fix is to constrain the shape of the answer and to measure cost per completed operation rather than per call.
- · 6 min read
Your Budget Cap Resets on Resume, Which Is Exactly When the Cost Arrives
A spend limit on an automated run is a control only if it accumulates across the whole task. On at least one widely used agent runner the cap applies per invocation, so resuming or continuing a task starts the budget again. Cost controls therefore stop working at the precise moment a task goes back for more work, which is when the spend happens. The fix is to track cost outside the tool, against the task rather than the invocation.
- · 8 min read
The Fan-Out Budget: Expand, Filter With Something Cheap, Spend the Paid Call Last
When one user request has to become many calls to a paid provider, cost and latency scale with the number of candidates rather than with the number of requests. The fix is an ordering: generate candidates from something you own, filter them against something cheap, and spend the expensive call only on survivors. The budget then holds per request instead of per candidate.