Pay for the work, or pay to be ready?

The problem, in plain words

Every cloud service has a cost shape made of two parts. The floor is what it costs doing nothing: the monthly bill at zero traffic. The slope is what each unit of work adds: each request, each second, each GB.

Services that scale to zero have no floor and a steeper slope. You pay only for the work, at a higher price per unit, and the first request after a quiet spell may have to wait while something starts. Services that are always on have a floor and a flatter slope. You pay to be ready, and each unit of work costs less once they're busy.

Neither is cheaper. Which one wins depends on the shape of your load, and the load has a shape you can read.

The picture

The same HTTP function, priced both ways. Lambda has no floor and climbs with every request in flight; the smallest highly available Fargate service starts at its floor and climbs slowly.

On the bill alone, the lines cross at about 2 requests in flight. The shaded band starts at 10, because below it the saving doesn't pay for owning a load balancer, health checks and deploys.

The options

Pay for the work. Lambda costs nothing between requests. A Fargate scheduled task bills only while it runs: a two-hour nightly job on 2 vCPU and 4 GB is about $5.70 a month, against about $69 for the same task left running. Neon scales to zero after a few idle minutes. A Runpod GPU stopped when idle costs only its volume. The price of all that is the wait: a few hundred milliseconds for Neon to wake, about 15 seconds for a paused Aurora (30 or more after a day), a capacity error from a SageMaker endpoint scaled to zero.

Pay to be ready. A Fargate service in its smallest highly available form (two tasks, two zones, a load balancer) costs about $73 a month before it serves a request. Aurora Serverless v2 kept at 0.5 ACU is about $73. An on-demand Kinesis stream is about $36, idle. In return, one container serves many requests at once, and nothing ever has to wake up.

Pay to be ready, in advance. Savings Plans and reserved instances buy that floor for one to three years at a discount, which is much bigger on containers than on Lambda.

The floors that hide in the plumbing. A NAT gateway is about $43 a month for every zone, before data. RDS Proxy is at least about $26 a month, and a Serverless v2 cluster behind a proxy never pauses. These floors don't show up on the service you chose. They show up on the bill.

A concrete scenario

A Sydney company runs a scheduling app for field technicians.

• Its API is busy from 7 am to 6 pm and nearly silent at night: about 5 requests in flight on average. On Lambda that's about $188 a month; the smallest highly available Fargate service is about $73. I'd stay on Lambda: $115 a month doesn't pay for owning a load balancer, health checks and deploys.
• Its nightly import runs for two hours, as a Fargate scheduled task: $5.70 a month. Then someone puts its tasks in private subnets with a NAT gateway in each of two zones: about $86 a month of floor around a $5.70 job, by my arithmetic. That's exactly the over-provisioning I warn clients about. The lean setup has no NAT.
• Its database is Neon. Development, staging and preview branches scale to zero. The production branch doesn't, because a technician opening a job at 6 am shouldn't wait for the database to wake.

In year two, a partner starts syncing all night. The API's floor at 3 am climbs to about 50 in flight, and its Lambda bill to about $1,883 a month, against about $437 on Fargate. Now pay to be ready: move that steady HTTP core to a container, keep Lambda for events, queues and spikes, and after three months of steady usage, cover the new floor with a Compute Savings Plan.

The trade-offs

The rule of thumb

Pay for the work until the floor is real. The signal is the load at 3 am, not the peak and not a high reserved concurrency, which costs nothing. Near zero at night: scale to zero. High all day: pay to be ready. And once it has been steady for three months, commit, but only to the floor: the lowest hourly spend you saw, never the average.

When the answer changes with scale

• Small and new: scale everything to zero. On a small bill, a floor is the biggest line.
• Spiky, at any size: stay. A function at 2 in flight that jumps to 300 for ten minutes a day costs about $153 on Lambda; a container sized for that peak costs about $1,960.
• Steady and growing: move one function when its overnight floor stays at about 10 or more in flight (or its average is 30% of its peak), the CPU is at most a quarter of each request, and its bill passes about $500 a month. By my arithmetic, a function's monthly bill divided by about $38 is its average requests in flight (1 GB, Arm).
• Must answer instantly: pay to be ready even at low traffic. That's what provisioned concurrency and an always-on database branch buy: latency, not price.
• GPUs: the answer flips when the GPU never stops. Rent by the hour for bursts; a 3-year Savings Plan puts an L4 at about $235 a month for inference around the clock (Runpod or AWS for a GPU?).