Model guides
Run open LLMs on Apple Silicon
Practical hardware guides: how much unified memory each model needs at each quantization, which Mac chip runs it, how fast it goes, and what it costs per month as a dedicated deployment.
Llama 3.1 8B
8B · Llama license · ~4.9 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
Qwen 2.5 32B
32B · Apache license · ~19.9 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
Llama 3.3 70B
70B · Llama license · ~42.8 GB at Q4_K_M
Runs on M4 Pro (Mac mini) from $199/mo.
Qwen 2.5 72B
72B · Apache license · ~44.1 GB at Q4_K_M
Runs on M4 Pro (Mac mini) from $199/mo.
Llama 3.1 405B
405B · Llama license · ~245.5 GB at Q4_K_M
Needs a Thunderbolt 5 cluster — see the guide for configs.
Mistral Large 2
123B · Mistral license · ~74.6 GB at Q4_K_M
Runs on M4 Max (Mac Studio) from $286/mo.
DeepSeek V3
671B MoE (37B active) · DeepSeek license · ~406.8 GB at Q4_K_M
Needs a Thunderbolt 5 cluster — see the guide for configs.
Llama 3.2 3B
3B · Llama license · ~1.9 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
Mistral 7B
7B · Apache license · ~4.4 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
Qwen 2.5 7B
7B · Apache license · ~4.6 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
DeepSeek R1 Distill 8B
8B (distill) · MIT license · ~4.9 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
Qwen 2.5 14B
14B · Apache license · ~9 GB at Q4_K_M
Runs on M4 (Mac mini) from $99/mo.
How these numbers work
Weight sizes are computed from parameter count × effective bits per weight (Q4_K_M ≈ 4.85 bpw, Q8_0 ≈ 8.5 bpw, FP16 = 16 bpw). Recommended memory adds 15% runtime headroom plus a KV-cache allowance for the model's context window. Tokens/sec figures are our M4 Pro baseline estimates — a public benchmark table covering every shipping chip is in the works.