Hardware compared
What to run local models on in 2026
The honest short version, and it is not the one you would expect from a company that rents Macs. If your model fits in 32 GB, buy an NVIDIA card. An RTX 5090 moves 1,792 GB/s against 546 for an M4 Max, and nothing Apple sells below the Ultra tier comes close. Above 32 GB the question changes completely, because speed stops mattering when the model will not load at all, and that is where unified memory earns its price.
Disclosure. We rent dedicated Apple Silicon, so we have an obvious interest in one of these three answers. That is exactly why the numbers below are computed the same way for all of them, why every specification links to its source, and why the section on where a Mac is the wrong purchase is the longest one on the page.
The market has split into three shapes
Fast and small. Discrete GPUs put GDDR7 behind a wide bus and reach between 936 and 1,792 GB/s. The catch is that consumer cards stop at 24 or 32 GB, so the models they can hold are limited to roughly 32B at Q4. Two cards double the capacity on paper but split the model across a bus, which is a different and worse thing than one pool.
Big and slow. The 128 GB unified memory boxes, the NVIDIA DGX Spark (GB10) and the AMD Strix Halo systems, use LPDDR5x. They will hold a 123B model, and they will do it at 273 and 256 GB/s respectively, which is between a quarter and a seventh of a modern GPU.
Big and fast, at a price. Apple’s Ultra tier is the only consumer hardware that is both. The M5 Ultra addresses 512 GB at 1,200GB/s, which is more memory than any of these and more bandwidth than all but the very fastest cards. NVIDIA’s answer in that quadrant is the RTX PRO 6000 at 96 GB and 1,597 GB/s, which is a workstation part with a workstation price.
The comparison that surprises people
The NVIDIA DGX Spark (GB10) costs $3,999 and has 273 GB/s of memory bandwidth. The M4 Pro, a two year old mid tier chip in a Mac mini, has 273 GB/s. They are the same number. The M4 Max (40-core GPU) has 546 GB/s at the same 128 GB of capacity, which is twice the Spark, and the M5 Ultra is 4.4 times it. If you are buying a 128 GB box for generation speed, that is the comparison to make before you spend the money. If you are buying it for CUDA, prompt processing, or fine tuning, none of the above applies and the Spark is a reasonable purchase.
Everything, at Q4_K_M
Tokens per second for generation, computed identically for every row from memory bandwidth and model size. A dash means the model does not fit in that hardware at all. Memory shown is the largest configuration available.
| Hardware | Memory | Bandwidth | 8B | 32B | 70B | 123B | Price |
|---|---|---|---|---|---|---|---|
| GeForce RTX 3090Discrete GPU | 24 GB | 936 GB/s | 105 | — | — | — | ~$1,300 |
| GeForce RTX 4090 | 24 GB | 1,008 GB/s | 109 | — | — | — | varies |
| GeForce RTX 5090 | 32 GB | 1,792 GB/s | 146 | 62 | — | — | varies |
| RTX PRO 6000 Blackwell | 96 GB | 1,597 GB/s | 139 | 57 | 30.1 | 18.2 | varies |
| NVIDIA DGX Spark (GB10)Unified memory box | 128 GB | 273 GB/s | 42.9 | 12 | 5.7 | 3.3 | ~$3,999 |
| AMD Ryzen AI Max+ 395 (Strix Halo) | 128 GB | 256 GB/s | 40.7 | 11.3 | 5.4 | 3.1 | ~$2,000 |
| M5 UltraApple Silicon | 512 GB | 1,200 GB/s | 120 | 45.5 | 23.3 | 13.9 | varies |
| M5 Max (614 GB/s) | 128 GB | 614 GB/s | 80 | 25.5 | 12.5 | 7.3 | varies |
| M5 Pro | 64 GB | 307 GB/s | 47.3 | 13.4 | 6.4 | — | varies |
| M6 | 32 GB | 153 GB/s | 26 | 6.9 | — | — | varies |
| M4 Max (40-core GPU) | 128 GB | 546 GB/s | 74 | 22.9 | 11.2 | 6.5 | varies |
| M4 Pro | 64 GB | 273 GB/s | 42.9 | 12 | 5.7 | — | varies |
| M3 Ultra | 512 GB | 819 GB/s | 97 | 32.9 | 16.4 | 9.7 | varies |
Before you spend three thousand dollars on a guess
Everything above assumes you already know which model you need. Most people do not, and that is the expensive part: the row you are looking at costs thousands, and you cannot return it once you find out that a 70B at 5 tokens/sec is too slow for the thing you wanted it for, or that a 32B would have been fine all along.
A month on a rented M4 Pro with 64 GB costs $286. That is enough time to run your actual workload, on your actual documents, at the speeds in this table, and find out which row you should be buying. Then buy it.
And the case against us, since it is the same page. If you already know what you need, the machine sits on your desk, and you will still be using it in three years, owning wins on hardware cost and it is not close. We work through that arithmetic in full on the buy versus rent page, including the parts people forget: a static IP, power, a residential uplink, and being your own on call at three in the morning. Renting wins on uncertainty and on access, not on price.
Where a Mac is the wrong purchase
- Anything that fits in 24 or 32 GB. A used RTX 3090 at around $1,300 runs an 8B model at 105 tokens/sec. The base M4 does 20.8. That is not a close call. The 3090 runs out of memory before 32B, but a 5090 holds it and still runs it at 62 tokens/sec, against 12 on an M4 Pro.
- Fine tuning. MLX is genuinely good and improving, but the ecosystem assumes CUDA. You will hit repositories that will not run without porting, and the time you spend doing that is usually worth more than the hardware difference. If training rather than serving is the point, this one consideration outweighs every number in the table above.
- Long prompts with short answers. Prompt processing is compute bound, not bandwidth bound, and NVIDIA hardware is substantially better at it. The NVIDIA DGX Spark (GB10) processes prompts two to three times faster than a Strix Halo box despite nearly identical generation speed. For classification or retrieval over whole documents, prefill is most of your wall clock time.
- Anything you want to serve to many users at once. These figures are single stream. Batched serving is where GPU compute and mature stacks like vLLM pull away, and none of the unified memory machines, Apple included, are the right tool.
Where a Mac is the right purchase
- Models between 32 and 128 GB, where you still want speed. This is the gap the GPUs leave. A 70B at Q4 will not load on a 5090; on a 128 GB box it runs at about 5.7 tokens/sec; on an M4 Max (40-core GPU) it runs at 11.2.
- The very large models, at all. Above 128 GB the field empties out. Apple’s Ultra tier at 256 and 512 GB is most of what is left short of datacenter hardware.
- Machines that have to be quiet, small, and always on. A Mac mini idles at a few watts and is silent. A 5090 draws hundreds of watts under load and needs a case and a power supply built around it. For a box that sits in a room with people in it, or in a rack you pay for by the watt, this matters more than the spec sheet suggests.
Where the newest Apple chips land
Apple moved the whole line up a tier in 2026, and it changed the comparison more than the reviews suggested. Figures verified 2026-09-11.
| Chip | Bandwidth | Max memory | Versus the DGX Spark |
|---|---|---|---|
| M5 Ultra | 1,200 GB/s | 512 GB | 4.4x bandwidth |
| M5 Max (614 GB/s) | 614 GB/s | 128 GB | 2.25x bandwidth |
| M5 Pro | 307 GB/s | 64 GB | 1.12x bandwidth |
| M6 | 153 GB/s | 32 GB | 0.56x bandwidth |
Read the M6 row before drawing the wrong conclusion from the others. Apple’s entry desktop chip sits at 153 GB/s, which is below both 128 GB boxes, so the boxes are not simply slower than Macs. The crossover is the M5 Pro, the first chip in a Mac mini to edge past them at all, and it does so by twelve percent. What separates the classes is not the entry point but the top: Apple keeps scaling bandwidth to 614 and 1,200 GB/s while the boxes stop at 273. Their argument was never speed. It is 128 GB for around $2,000, and against that specific number Apple has nothing to offer.
How these numbers were produced, and what is wrong with them
Every speed on this page comes from one formula applied to every row: seconds per token equals the weight size divided by bandwidth times 0.915, plus 3.9 ms of fixed overhead. Those two constants were fitted by least squares to six models we measured ourselves on a base M4, and they reproduce all six to within 4%. The method and the raw data are on our benchmarks page, and the calculator will run it for any combination you like.
The obvious objection, stated before someone else does. That overhead constant was measured on Apple hardware running Ollama. Applying it to CUDA and ROCm assumes a comparable per token cost, which we have not verified and cannot until we measure those stacks. It is not a rounding error either: without it the formula claims an RTX 5090 generates over 300 tokens/sec on an 8B model, which nobody observes. With it the figure is 146, which matches what people report. We think that is the better assumption, but it is an assumption, and the faster the hardware the more the answer depends on it.
Everything else here is a published specification rather than a measurement. Sources for each piece of hardware:
- GeForce RTX 3090 — 24 GB at 936 GB/s (used, $1,200 to $1,400 as of late 2025). Source
- GeForce RTX 4090 — 24 GB at 1,008 GB/s (used, varies widely). Source
- GeForce RTX 5090 — 32 GB at 1,792 GB/s (retail, varies). Source
- RTX PRO 6000 Blackwell — 96 GB at 1,597 GB/s (workstation class, quoted rather than listed). Source
- NVIDIA DGX Spark (GB10) — 128 GB at 273 GB/s (list price at launch). Source
- AMD Ryzen AI Max+ 395 (Strix Halo) — 128 GB at 256 GB/s (128 GB systems from a little over $2,000). Source
Street prices move constantly and we do not try to track them. The memory and bandwidth columns are the durable part of this comparison; treat the price column as an order of magnitude.
Common questions
Is a Mac or an NVIDIA GPU better for running local LLMs?
It depends entirely on whether your model fits in the GPU's VRAM. Below about 24 to 32 GB a discrete NVIDIA card wins and it is not close: an RTX 5090 moves 1,792 GB/s against 546 for an M4 Max, so it generates roughly three times faster on the same model. Above that ceiling the comparison inverts, because a 70B model at Q4 simply will not load on a 32 GB card while a 64 GB Mac runs it. Capacity decides what is possible and bandwidth decides how fast, in that order.
Is the DGX Spark faster than a Mac for local inference?
For generation, no. The DGX Spark has 273 GB/s of memory bandwidth, which is exactly the same figure as an M4 Pro and half that of an M4 Max. Since generation speed is set by bandwidth divided by model size, a Mac Studio generates about twice as fast at the same model. Where the Spark wins is prompt processing, which is compute bound rather than bandwidth bound, and the CUDA software stack, which matters enormously if you intend to fine tune rather than only serve.
What is the cheapest way to run a 70B model locally?
A 70B model at Q4 needs roughly 50 GB of memory once the context and the operating system are accounted for. The cheapest hardware that holds it is a 128 GB Strix Halo system at a little over $2,000, which will generate around 5 tokens per second. A 64 GB Mac reaches a similar speed. No consumer GPU holds it at all without splitting the model across two cards.
Why do unified memory boxes have so much less bandwidth than GPUs?
They use LPDDR5x, which is system memory designed for power efficiency, while GPUs use GDDR7 or HBM designed for throughput. That is the whole trade. LPDDR5x lets a machine address 128 or 512 GB in one pool at moderate speed; GDDR7 gives enormous speed across a pool that stops at 32 GB on consumer parts. Apple's Ultra chips are the exception that proves the rule, reaching 1.2 TB/s from LPDDR by using an extremely wide bus, and they are priced like it.
When the answer is not a box on your desk
There is one case this whole comparison cannot settle, because it is not about hardware. If the machine has to be reachable by your team or your application, around the clock, from anywhere, then buying the fastest box is the easy half and the hosting is the rest of it: a static IP, an uplink that does not drop, disk encryption, and someone awake when it hangs. We rent dedicated Apple Silicon from $99/mo with that part already done, Ollama installed and an OpenAI compatible endpoint exposed. What we own today is M4 generation; the M5 and M6 machines are listed in our configurator at real prices and route to a waitlist rather than to checkout, which is how we decide what to buy next.