GPU
Renting a GPU by the month instead of by the hour
Hourly GPU billing is built for bursts. If your model runs all day, the arithmetic changes completely. How to work out which shape fits your workload, and the VRAM maths that decides which card.
Hourly GPU rental is priced for bursts. You spin an instance up, run a training job, shut it down, and pay for the hours you used. That is exactly right for a fine-tune that runs twice a month. It is a poor fit for anything that has to answer requests at 2pm on a Tuesday, and the reason is arithmetic rather than any vendor being unfair.
The break-even is about 730
Take any hourly GPU rate and multiply by 730, the average number of hours in a month. If you are running that instance continuously, the monthly equivalent is that number. Many teams budget from the hourly figure as if they will use a quarter of the month, then discover the inference endpoint they built is, by definition, always on.
The honest way to decide is to write down how many hours a month the card will actually be doing work:
- Training and fine-tuning. Genuinely bursty. Fifty to a hundred hours a month is common. Hourly is usually right, and monthly usually wastes money.
- Always-on inference. A product endpoint, a chatbot, an internal tool people use during the day. This runs 730 hours a month by definition.
- Batch processing on a schedule. Nightly embeddings, nightly re-ranks. Perhaps 200 to 300 hours a month depending on how long the job takes.
- Development and experimentation. Erratic. Some weeks heavy, some idle.
The middle two categories are where a fixed monthly price wins, because you are not paying for the flexibility you are not using. The first category is where hourly wins, and no amount of monthly discount changes that.
VRAM decides the card, before price does
People shop for a GPU by price and then discover the model does not fit. Video memory is the hard constraint, and it is worth doing this sum before comparing any prices.
A model's weights, stored in half precision, take roughly two bytes per parameter. That means:
- A 7 billion parameter model at fp16 needs about 14 GB just for weights.
- At 8-bit it needs about 7 GB.
- At 4-bit it needs about 3.5 GB.
Then add the KV cache, which grows with context length and concurrency, and add the runtime's own overhead. A useful rule of thumb is that usable VRAM for a model is roughly the card's VRAM minus 1 to 2 GB.
So a model that is described online as "runs on a single GPU" may need a 24 GB card at fp16 and fit comfortably on an 8 GB card at 4-bit quantisation, with a quality difference you should measure rather than assume. This is why a cheap 8 GB card is not a toy and an expensive card is not automatically the right buy.
| Card | VRAM | Realistic fp16 | Realistic 4-bit |
|---|---|---|---|
| RTX 2060 SUPER | 8 GB | 3B and below | 7B |
| L4 | 24 GB | 7B | 13B to 30B |
| RTX 4090 | 24 GB | 7B | 13B to 30B |
| A100 | 80 GB | 30B to 40B | 70B |
Those bands are guidance, not guarantees. Quantisation quality varies by model, context length eats VRAM quietly, and batch size multiplies the cache. Test with your actual model and your actual prompt lengths.
What "a whole card" changes
The distinction that matters more than hourly versus monthly is whether you get the whole card or a slice of one.
On a shared slice, your throughput depends on what the other tenants are doing at that moment. That is fine for development and miserable for a latency-sensitive endpoint, because you cannot explain to a customer why the reply took four seconds. On a dedicated card, the performance you measured is the performance you get, and the numbers are reproducible.
If you are choosing between a cheaper shared slice and a dedicated card, the question to ask is not "which is faster on average" but "which is predictable". For a product, predictability is usually worth more than the average.
The charges that are not the GPU
Hourly billing on a large cloud provider rarely bills only the compute. Watch for:
- Egress, which is charged per gigabyte in most regions. A GPU that streams images or serves a model API can move a surprising amount of data.
- Storage, billed separately per GB per month, and often duplicated when you keep a snapshot.
- Idle instance charges. A stopped instance may still bill for attached disk.
- Currency conversion, which is invisible on the price page and depends entirely on your card.
A monthly price in rupees is simpler to model precisely because there are fewer lines to model. It is not automatically cheaper. It is easier to predict, and for a budget that has to be approved, predictability is its own kind of value.
When hourly is simply correct
A weekly fine-tune job, a two-day benchmark, a hackathon weekend, a model evaluation that runs once a quarter. In all of those, paying for 730 hours of a machine you use for 40 is just waste, and the flexibility you are buying is real.
The mistake to avoid is buying hourly for something that will be permanently on, or buying monthly for something that runs once a month. Both are the same error in opposite directions: paying for flexibility you are not using, or buying a commitment you cannot use.
What to do before you commit
- Write down the hours per month the card will genuinely do work.
- Work out the VRAM your model needs at the precision you intend to run, including context and batch.
- Pick the smallest card that fits, not the largest you can afford.
- Measure latency and throughput on your real prompts before you scale anyone's expectations.
- If the number of hours is high and steady, price the monthly option. If it is low or erratic, price the hourly one.
That is the whole decision. Everything else is marketing.
- gpu
- inference
- cost

