It was, unfortunately, a nightmare. There were tons of tickets and even threats of lawsuits from customers whose service got cut off hard at the worst possible time due to organic growth/going viral/big event/nobody knew about the limit/etc. Not only did they lose all the leads and revenue they would have gotten from that bump, but they also pissed off their own existing users who suddenly couldn't use the service either.

Generally speaking, it's much better to use alerts instead of hard limit. Even in the worst case (hackers pwn your credentials and mine Bitcoin or whatever) the rest of your business is unaffected and you can negotiate with the billing department at comparative leisure.

This is all assuming you have humans operating the service. If you're letting AI agents yolo infra in prod, you have a whole series of new problems.

Queue lengths, request sizes, response wait duration, message payload size, authentication attempts, allocation rates -- there's always some upper number beyond which the system is so messed up you'd rather it crashes.

An argument against this is that businesses don’t want their hosted applications to start throwing errors because some budget was exceeded. I expect that most businesses and individuals would prefer errors to a surprise $10,000+ bill.

Indeed. If you want a surprise $10,000 bill that's still not an argument against a hard cap -- just set it at $9,999,999 instead, or wherever you don't want the surprise bill. There's always a number that indicates something has gone insane. There's always a sensible upper limit to any operation.

When video models just came out, I wanted to make a short clip. Gemini didn't have enough control back then, so I started in Studio. I had $10 on my account. Try, try, try, not good, retry, altogether maybe 20-30 retries with 4 choices for a 10 second video.

Wake up next morning to an email from Google - my Studio account is frozen because of negative balance.

I check it, it's at -$160. Not financial ruin, but a painful sum for a 10-second video I didn't even use in the end.

I don't think there was (or is) a setting in there that says "stop everything when I'm in the negative".

Almost a year later, it's still as bad. Trying to make some music via Lyria, I need to wait 10-12 hours for the API charge to show in my costs. Now I had the bitter lesson, I wait after every major novel experiment to see how much it costed me.

Coding is solved, my ass.

I can subscribe to your service for a specific fee on a monthly basis ($20/month say), and take the risk of losing that month's fee if your service or I make mistakes, or I can choose to drop another $20 mid-month, or anything for convenience.

Saying that the computer will "control" the billing and can run haywire tells me that I don't want to be anywhere near your pile of bad incentives.

It’s such a basic thing, not having it has to be deliberate to make you accidentally spend more than you’d like.

I actually think network ACL triggers based on billing might be the only way to really enforce this.

I witnessed a DDoS attack once that changed how I think about billing. It was locally provisioned hardware and the attackers had saturated the switches. Naively I said "just block the CIDRs" but the problem was the incoming ram is so saturated that it can't even get to the point of "deny" in the firmware.

So from a technical perspective if there's an internal DDoS at AWS what do you do? Do you turn off the endpoint? Do you drop the sources from hitting it at the router? And even that costs money. Anyway that incident gave me a different level of appreciation for this challenge.

Edit: this is mainly targeted at the people complaining why this took so long. At some point in scaling even telling you "no sorry" in a nice way is expensive. I'm sure recruiters can sympathize with this nowdays.

Also, does anyone know why it’s taken this long? I suspect it’s a technical reason. While one could be cynical, I doubt it’s an intentional business/product decision. Hard spending caps are both excellent product differentiators and could possibly save these providers money as they don’t have to forgive their users when they accidentally over-use a service.

https://cloud.google.com/blog/topics/cost-management/new-ear...

Edit: Ugh it's fake. Literally only works for four random services, unsupported for all the rest. Completely useless for all of my projects. Also dumb that the only supported term is "monthly" considering that months are different lengths, and that they don't bother to account for credits or discounts. Maybe next decade they'll get around to implementing something useful.

Even if we have a negotiated yearly contract for $X spend per year, maybe we’ll hit that spend in 6 months instead of 12. Having some kind of automated telemetry saying how we’re trending would be so useful.

I’ve gotten vague warnings from customer success people saying vaguely that, but without any warning of what’s truly happening. The more we abstract away from money (tokens, credits, etc), the more we need a way to translate right back to money, to see how close we’re getting to any limits over time.

I don’t want to find out in month 5 that the contract which was expected to cover a year is now going to run out in 14 days.

Phone service, bank card, home internet etc.

If you don't pay your bill than they just cancel your membership and it works ok.

People in western countries are just getting shafted by companies for (mostly) no reason because an alternative balance is just inconceivable.

In this proposed world where this static software is replaced with dynamic software ("AI will build it JIT"), doesn't that destroy the value proposition?

And in the world where normal software becomes AI-integrated ("software has intelligence "), isn't this tantamount to very very high hosting costs - where each execution now costs centi-dollars instead of nano-dollars?

Seems like the only use case where the cost model is equal or better is "AI writes the software".

I know AWS and similar sites have no such concepts, but other than those, this is an existing option of most kinds of service providers.

Realistically, most providers can't even allow you to consume $10k usage if they don't have the certainty that you can pay up. Prepaid is what gives them that certainty.

The whole idea was insane. We shunned paid compute services in favour of personal computing and then sold our souls back again.

It should be possible to open my bank app, see all my recurring payments (subscriptions), and be able to cancel them with one button. And this should count as an official termination.

Also, limits, etc.

The entire MCP ecosystem is ludicrous. You’re paying for inference for an agent to make the same decisions over and over again, and yet the actions they’re taking can be so easily written by those same agents into a bash script you can run again and again, deterministically and for free.

MCP is the problem. Having agents “use a product on your behalf” is the problem.

The pattern you’re looking for is that agents should write scripts that use products for us, and that doesn’t need a new protocol.

And everywhere in Europe you have clear price sheets (unlike the deliberately opaque mess of the US price sheets with more small-print than a packet of pills), which means even if you are at an EU provider with no hard caps you can still accurately reason and predict your costs.

Just a few examples....

Cloud providers:

    - Upcloud
    - Exoscale

    - Verda
    - PrivateMode

The "viral" bit is easily solved through basic monitoring of metrics that everybody should be doing. I believe the cool-kids give it the fancy name of Site Reliability Engineering (SRE). All you need to do is top-up your balance / adjust your cap if your metrics are trending upwards for an explainable reason. Its not rocket science.

As for the "too difficult" that's just a lie. It just suits the US cloud providers better to have you spend spend spend on their messy soup of random interdependent microservices.

I've had an AI coding agent enter a doom loop for hours multiple times now. having a budget limit for my openrouter key is helpful, but it's already spent then.

I've been building a thing you can hook into ai workflows that uses multiple detectors if an agent is starting to loop and will then send a kill command to that chain. I don't have any testers for it though.

One solution is banking apps that let you create extra cards and assigning hard limits to spending. Wise & Revolut apps can do that.

Have to mention OpenRouter also, I logged into OpenRouter through pi, on the auth screen it optionally let me set a limit which is very smart and user friendly.

I realized, what am I actually afraid of. Well, overspend. So I just set then all to disable auto-reloading. Now if it blows up, I'm down $5.

Same story with containers. Just give it root on a VPS, and if it blows up, I'm down $3.

I had a spending limit on for $30, so why did it keep charging? Because the spending limit is meaningless without a hidden checkbox called “enforce spending limit,” which is (or at least was for me) off by default.

To OpenAI’s credit, they refunded the money.

Was a 6 month nightmare proving we did nothing wrong.

In an ideal world, our agents could help with this.

Not exactly the sort of case Simon has in mind, but I tell Claude to keep to hard daily limits on its OpenRouter spending for two long-running projects [1, 2].

A Routine for each project fires ever few hours, and Claude decides itself what to do in each session. It does tasks that require calls to other models through OpenRouter only when it is still within its daily budget for that project; after it reaches that cap, it does other tasks that don’t require extra spending.

[1] https://github.com/tkgally/je-dict-1

[2] https://github.com/tkgally/eex-dict

A regular AWS account doesn't have the notion of "project" and Cost Management doesn't have those spend limits.

It’s not in their interest to make cost controls work well.

Ideally you’d be able to set something granular like “allow this service to scale up only 10x, measured at an hourly level, and alert me when it happens. Drop all requests that exceed 10x”.

And then you are mostly in a throttling situation until the burst clears or a human can review & accept increased usage is ok/increase thresholds. I’d rather have services go slow during excess load (like a real server) than go dark for remainder of month.

This seems a lot better than brute force “turn everything off at $X level of monthly billing” or “no limits you can charge me infinity dollars”.

Just last week I was looking for similar functionality in Cloudflare as I am exploring publicly exposing apps I have built (as opposed to just me and some friends on a home lab).

My main concern on public cloud platforms is costs. I never want to spend more than, say, 10 EUR on a simple service. In my mental risk matrix, the chance of a cost-related issue has been increasing more-and-more. I am creating and hosting more than ever, and the chances of malicious activity/abuse are in my opinion higher than ever before.

It feels like cloud providers either are unaware of this, or riding a wave of increased turnover - which I think a more likely.

I had a personal development account for ~15 years. I tinker with infrastructure stuff and had built some centralized event reporting. One day about two years later I turned on sqs data events into cloudtrail. What I didn’t realize was that this closed a feedback loop and over the next couple of hours my run rate went to about $4k per day in cloudtrail+sqs usage.

I didn’t realize it until I hit the next months billing alarm immediately the next month. I’d racked up $25k in usage fees.

I’ll be using this feature. Nothing I run is worth that risk.

I get it, the problem is definitely worth solving for. Waking up with a $100k bill isn’t great.

At the same time, from a product perspective the proposed solution might be a bad idea. Simply having hard caps as default will definitely turn out to be as bad for some people as a $100k bill, see for example (1).

You can’t come up with good product changes if you don’t discuss the potential negative effects of a change.

(1) https://smackernews.com/item/49950196 HN

Turns out it's impossible to do. You have to delete your entire account.

Hobbyist needs to not get wiped out. Business need stuff to stay online.

In order to learn how to use the platform, employees in business and startups need a relatively safe space to learn too so even they have mixed needs.

Big tech is definitely the worst offender though on handing out footguns and relying on "beg support for mercy" model.

Seriously. You all asked for this.

You can be down serious money even with a limit set.

Someone told me some years ago that it was "impossible" for Google to apply an upper usage caps due to technical design reasons which I found absurd. You can build a globally distributed continent-scale SQL database with consistency guarantees, but you can't stop me paying more than X USD when my usage ticks over that? Huh?

It smelt much more to me like Google's business model made it impossible to stop customers spending money, not their engineering.

I voted with my wallet.

weird, that

AWS, is a loot box... Tokens are just in game currency, and that sales person is just metrics that have identified your spending as making you a whale.

Your average CTO from the last decade turned a fixed cost into variable spending that looks like a mobile game.

Nobody wants to wake up to an email sent at midnight warning about a budget limit and find that, while they slept, their rogue service had consumed several hundred (or several thousand) more dollars of usage.

Monitoring and a circuit breaker. If you are given the tools, make your own heuristic and flip the breaker yourself. Don't let someone else turn it off, and be hostage to their process failures for getting it back on. By self-selection, if you are a hard limit customer, you are not front of their service line.

An argument against this is that businesses don’t want their hosted applications to start throwing errors because some budget was exceeded. I expect that most businesses and individuals would prefer errors to a surprise $10,000+ bill.

Based on a career serving the Fortune 50, CTO for trillion dollar bank, etc.: "No." Assuming actual business is being done by the machinery in question...

The lights must stay on.

Cost management cannot shut off the enterprise. Cascading costs, even before reputation, are incalculable.

But the idiocy is incredible, even allowing for this to be happen in a business is so infantile that the only hard cap that should be important is not to allow stupid people in the machine room.

Cheers!

Could not agree more, this needs to be table stakes for any usage based service.

What on earth is Simon whittering on about?