What hardware to buy to rent out GPUs for AI workloads

The question reaches us in two forms: «what should I buy to rent out a GPU for AI workloads» and «is the machine I already own good enough». This article answers the first one. The goal is a build around ~1M rub that maximizes income and passes vast.ai verification without hitting the known traps. Every measurement below comes from our own fleet: TVT (4× RTX 4090, EPYC 7C13, Supermicro H12SSL-i), tv5080v (2× RTX 5080, EPYC 7K62, TYAN), tv50 (RTX 5090, Ryzen 9 9950X) and three more machines, one of them a client's. Per-machine income is public on the fleet page of our site Vast Sale: our live fleet page

The order of operations: arithmetic first, shopping cart second

The typical mistake is to start by picking a card. Income works out as: number of GPUs × your hourly rate × 730 hours × utilization, and you keep the full rate — the platform's markup is added on top of what the client pays and is never deducted from your side. A new host sees 30–50 % utilization, a verified one 50–70 %, and that figure — not the card model — moves the result more than anything else. Run your own build through the calculator at the income calculator; the full cost and payback breakdown lives in our server economics article at our income breakdown card by card.

Two constraints must be checked before you buy anything, because either one can cancel the purchase outright. First, power: a 4× RTX 4090 rig on a server platform peaks at 2.2–2.4 kW and holds 1.8–2.0 kW under sustained ML load, which means a dedicated 16 A line with its own breaker, not a household socket. Site and electrical work are covered at how much power a server draws. Second, the link: we can measure your upload during a free consultation, and the platform's requirements plus the traps of Russian ISP plans are covered in what network a host needs.

Key criteria

• GPU — demand and $/W: bet on RTX 4090 / 5090. By the catalogue median on 25.09.2026 a host receives $0.37/h for a 4090 and $0.47/h for a 5090; lower-tier cards pay off through a cheap purchase price, not through the rate
• RAM — 32–64 GB per card, ECC RDIMM on a server platform. Low RAM = fewer rentals: some images simply fail to start, and VM mode takes 80 % of RAM multiplied by the share of rented cards (whole machine — 80 %, one card of two — 40 %) and demands the same amount free
• Disk — NVMe SSD: from 500 GB per server plus 300 GB per additional card. The disk is also a product of its own: storage sells at $0.10–0.20 per GB per month and keeps earning even on a stopped instance
• PSU with 20–30 % headroom, 80+ Gold/Platinum, sized from the whole-rig peak rather than from the cards: 4×450 W is 1.8 kW, while the rig peaks at 2.2–2.4 kW
• Cooling — an open-air frame with directed airflow, GPUs kept below 80–83 °C. Almost all the power turns into heat: a 2 kW rig warms a room like two space heaters
• NIC — judge by measurement, not by vendor: cheap gigabit Realtek parts (RTL8111/8168) on PCIe x1 cap upload at ~330 Mbit, while modern Realtek 2.5GbE and Intel chips deliver an honest link in our fleet. Measurements are in the table below
• CPU — many-core and with enough PCIe lanes: EPYC / Threadripper / Ryzen 9. The cores are not for computing but so that four tenants do not fight over one thread; the lanes are so that cards do not end up on x1
• Motherboard — working IOMMU and an ACS option in BIOS, otherwise VM mode cannot be enabled later, and that is a noticeable share of demand

GPUs: the price gap is wider than the income gap

The most common expectation is that a new card will earn several times more than the previous generation's flagship. Our August (31 days, 90 % average utilization) does not confirm that.

Do the math: $376 against $227 is a factor of 1.66, not two and not three. The client pays for an hour of rental, not for FLOPS, so in return per invested rouble the newer card does not make up the difference in purchase price. What it does give is more money per slot in the chassis and per kilowatt — and slots and kilowatts are finite. Hence the practical rule for a build: for four cards take 4090s, for two take 5090s. The full per-machine breakdown of our month is at our income breakdown card by card.

A caveat on the table: these are our rates and our plans for specific machines, not the ceiling of a card. We deliberately keep the pair of 5080s at the top of the market range — $0.25 an hour now against a $0.23 median — because we are rebuilding that machine soon and a month-long contract would be in the way — hence its modest row. Forecast a new build from the catalogue median, not from our numbers.

RAM, disk, PSU: concrete numbers instead of «some headroom»

A separate word on memory — the most underestimated line in the budget. VM mode hands the guest 80 % of RAM multiplied by the share of rented cards (all 80 % when every card is rented), and the platform's agent requires the same amount free in the output of free, page cache not counted. On a 16 GB machine that is 12.7 GB, and after a few docker pulls only 11.8 GB remains free: the VM fails to start with «insufficient free memory» even though the memory is formally there. Dropping caches fixes it on the spot, but the decision is made at purchase time.

The NIC: judge by measurement, not by vendor

This is the one part of a build where we advise trusting neither reviews nor the label on the box. Here is what our fleet showed.

Three builds for about 1M rub

• A — one powerful 4× RTX 4090 rig (~950k rub): maximum income and simplicity — one platform, one link, one power line, one set of pitfalls. Concentration risk: any failure stops all the income. VM is available after D3cold hardening
• B — two 2×(2× RTX 4090) rigs (~1M rub): diversification — a failure takes half the income instead of all of it. Pricier (two platforms, two PSUs, two power lines) and harder to maintain: everything you do by hand, you do twice
• C — 2× RTX 5090 (~820k rub): top demand and the best income per slot, VM possible, and about 180k rub left over — for a third card, a proper UPS or spare parts

How we choose between A and B. A first machine you assemble yourself is A: one platform means one tunnel, one link, one set of firmware and one procedure to learn. The diversification of B pays off once that procedure is established rather than still being learned. C suits sites limited in power or space: two cards instead of four, with comparable demand and half the heat.

Motherboard and VM mode: what to check before paying

VM mode means renting out a virtual machine with a card passed through. It raises the rate and thins out the competition, but it needs a board that can separate cards into their own IOMMU groups. You can only verify that on an assembled server, so before buying you read the board's documentation and other people's experience. Installing the OS step by step is covered at installing Ubuntu for Vast.ai, and the check itself takes three commands:

ls /sys/kernel/iommu_groups | wc -l        # 0 = IOMMU выключен в BIOS
lspci -nn | grep -i nvidia                 # PCI-адреса и ID карт
for d in /sys/kernel/iommu_groups/*/devices/*; do echo "${d#*iommu_groups/}"; done | sort

Three things from our practice. On tv5080v the ACS option was not where people look for it: AMD CBS → NBIO, a separate tab, and in that BIOS you save with F4, not F10. There is no AER option there at all — it is hidden in Auto and works. On tv80 a power cut reset the CMOS, VT-d went off with it, and VM clients stopped starting without a clear error; the very first line of the block above catches this — if there are zero groups, the driver is not the problem. And third: on a multi-GPU machine the platform's own check sees four identical 4090s as one, so its verdict should not be trusted blindly — we verify cards by UUID.

And about the RTX 4090 and its reputation. It has no «incurable reset bug»: the FLR bug belongs to Blackwell cards. A card can indeed fail to come back from passthrough if it drops into deep sleep (D3cold) when detached from the guest — only a cold reboot recovers it, and the number of cards in the build has nothing to do with it. The cure is hardening: device rules that forbid deep sleep. After that VM works: on a 4×4090 rig we validated the hardening with a controlled ladder of 1 → 2 → 3 → 4 cards in VMs without a single reset, and VM has been on since 20.09.2026. The 50 series needs -open drivers, the whole nvidia stack on apt-mark hold and nvidia_drm.modeset=0 pcie_aspm=off in GRUB. The whole procedure for enabling VM on a multi-GPU build — from the admission conditions to the first client — is laid out step by step here: VM mode on a multi-GPU host

Where we got it wrong

A dead memory module. On the 4090 rig the machine kept falling over while memtest stayed silent — it passed clean. The fault could only be proven by moving modules between slots and enabling swap: under swap the pattern followed exactly one module, and the slot itself turned out to be fine. Two purchasing lessons from that: buy identical modules from one batch, otherwise swapping them proves nothing, and buy one spare up front — it costs less than a day of downtime on four cards.

We trusted the NIC's vendor. A gigabit plan, a gigabit card, 330 Mbit of upload. While we looked for the problem at the ISP, the machine was losing search placement to symmetric nodes. Now the order is reversed: measure first, conclude second.

We did not check free space before installation. On a client machine the image cache grew to 46.6 GB and evicted the bandwidth test image: the platform's test started with a two-minute pull and failed — the only failure in that machine's whole log. The disk was not small; the free space was, and in a budget those are two different things.

We planned on traffic income. We set $3 per TB, a VM-mode client pushed 8.32 TB in 37 hours — and the report says Upload $0.00 / Download $0.00. The platform does not bill traffic for virtual machine rentals at all, while a neighbouring docker machine is billed normally. The lesson for a build: a fast link pays for itself through utilization, not by selling traffic, and a VM node with a fat link additionally attracts people who want transit rather than a GPU.

We picked a VPS by the wrong criterion. Our tunnel runs through a foreign VPS, and downloads from Docker Hub were not limited by our own link: traffic is throttled by the hosting provider's autonomous system — OVH and M247 were throttled for us, Private Layer was clean (and in October 2026 VEESP started getting cut: the list changes). The tunnel configuration had nothing to do with it, and we wasted time adjusting it. A tunnel provider should be chosen by measurement, exactly like a NIC.

What goes wrong after the purchase

This is no longer about choosing hardware, but it is worth knowing before you buy: downtime costs money, and here is where ours came from.

• The platform's controller gets stuck: the machine runs while the exchange considers it inactive. On 25 August four of our rigs went down that way (similar dropouts in September turned out to be the agent hanging on a call that bypassed the tunnel — a different cause and a different cure). The cure is resetting the file /var/lib/vastai_kaalia/controller_connection
• The platform itself wipes the VM flag: an hourly cron job removes the VM capability marker when all of a machine's cards are handed to virtual machines — the check looks for a drm/renderD* node, which a passed-through card does not have. Because of this tv5080v stood without VM for ten days, from 11.09 19:22 to 21.09 17:04
• The machine agent hangs on a call to the platform's web API (VerifySendMachInfo) — a blocking HTTP request with no timeout. From the outside it looks like a healthy machine that stopped sending heartbeats
• The host's network stack can take down a home router: our xray started before the DHCP address arrived, the packet to the VPS fell into its own tunnel, and 28 thousand sockets filled the router's conntrack table — on 20 September that triggered a false failover of four rigs to the backup VPS
• A CMOS reset turns off VT-d, and VM clients stop starting without any meaningful error message

None of these problems resolve on their own. The stuck controller is dissected at the stuck controller write-up, the backup tunnel and failover to a second VPS at the backup tunnel, and the diagnostic commands at our Linux command cheat sheet.

A checklist before you pay

• Income calculated from the catalogue median at 30–50 % utilization, not from someone else's 90 %: the income calculator
• Upload measured and compared against the requirement of 2.6 Mbit/s per 1 GiB of total VRAM
• The site's available power is known, and a dedicated 16 A line per server exists or is planned
• IOMMU and ACS found in the board's documentation — if you plan to run VM mode
• RAM: 32–64 GB per card in identical modules from one batch, plus one spare
• NVMe with headroom: 500 GB per server plus 300 GB per additional card
• PSU sized from the whole-rig peak, not from the sum of the cards
• Decided where the machine will sit in summer and where the hot air will go
• VM rental traffic left out of the income model — it is not billed