A GPU for models and experiments
I need an NVIDIA GPU to run or train open models, for an experiment or for a client's system.
My stack for this
- 01Runpod
Research and experiments: the card you need in minutes, billed per second, stopped at night for the price of its storage.
- 02SageMaker
When the data is the client's: training jobs and endpoints inside its account, Sydney included.
- 03EC2
A full GPU VM in the client's account, or inference running 24/7 on a Savings Plan.
- 04Hostinger VPS
The always-on box beside the GPU, for the agents and everything that needs no GPU.
Don't
- Fargate scheduled tasks
Fargate has no GPUs, for tasks or services, and neither has Lambda.
Doesn't cover
An LLM inside an app: call a model behind an API instead of renting a GPU. That's its own use case: An AI chat or LLM feature.
Questions that change the answer
Whose data is it? The client's, in its own account or in Australia, means SageMaker or EC2 on AWS.
Does it run in bursts or 24/7? Bursts point to Runpod, 24/7 to an AWS Savings Plan.
How much GPU memory does the model need, and does the code expect a full VM (its own Docker, systemd, UDP)? That decides the card, and whether a container will do.
The picks, in brief
What each one is, my verdict, and what it costs. Every name opens the full entry: when to use it and when not, the gotchas with their fixes, provisioning and alternatives.
My GPU for research and experiments: the NVIDIA card I need in minutes, billed by the second, stopped at night for the price of its storage. Not the cheapest GPU there is, and a container, not a VM.
Pricing: Secure Cloud, per second: RTX 4090 (24 GB) $0.89 an hour, L4 $0.59, RTX A6000 (48 GB) $0.59, L40S $1.09, H100 $3.99. Network volume $0.07 per GB a month, also billed while the pod is stopped. No fees for data in or out. Checked 1 November 2026: Secure prices rose 15–36% between July and October (the 4090 was $0.69), spot pods are retired, and savings plans were withdrawn in October.
My AWS answer for GPU work on a client's data: training jobs that bill only while they run, and endpoints that scale, inside the client's account, Sydney included. About 40% over the same EC2 box.
Pricing: us-east-1, on demand: ml.g6.xlarge (L4, 24 GB) $1.127 an hour, ml.g6e.xlarge (L40S, 48 GB) $2.61, ml.p5.4xlarge (H100) $7.91; Sydney ml.g6.xlarge $1.465. That's 40% over the same EC2 instance for the 24 and 48 GB cards, and 15% for the H100. Billed per second. SageMaker Savings Plans take up to 64% off. Checked 1 November 2026.
Servers you run yourself. For GPUs, special hardware or a big steady fleet; for a plain container, Fargate costs the same.
Pricing: Sydney, on demand: c7g.large (2 vCPU, 4 GB) $0.0944 an hour, about $68.91 a month; m7g.large (2 vCPU, 8 GB) $0.102 an hour, about $74.46; t4g.small $0.0212 an hour. Fargate on Arm with 2 vCPU and 4 GB is $0.09474 an hour, about the same. Compute Savings Plans take 24% (1 year) to 48% (3 years) off m7g. GPUs (checked 1 November 2026): g6.xlarge (L4, 24 GB) $1.0464 an hour in Sydney, $0.8048 in us-east-1; g6e.xlarge (L40S, 48 GB) $1.861 in us-east-1, not offered in Sydney; p5.4xlarge (one H100) $8.944 in Sydney.
Unbeatable price for an always-on box. CPU caps and no serious GPUs.
Pricing: KVM plans from 1 vCPU / 4 GB (KVM 1) to 8 vCPU / 32 GB (KVM 8). Intro prices $6.49 to $25.99 a month paid upfront for the term, renewing at $11.99 to $49.99 a month. Weekly backups and a dedicated IP included.
Where they sit
The layers of the stack this job touches, with every tool named on this page.
Compute
Where business logic runs.
Networking and hosting
Private access, exposure, DNS, servers.
Want this stack in your AWS account?
Cloudless Labs is a Sydney consultancy. We migrate on-premises systems to AWS, build serverless apps on this stack, and coach teams to ship with Claude Code and Codex.
Talk to us