Run Mistral Medium 3.5 on Apple Silicon
Mistral Medium 3.5 is a dense 128 billion parameter model with a 256K context and image input, released as open weights on 28 April 2026. At Q4_K_M the weights are about 78 GB, so a Mac Studio M4 Max with 128 GB runs it; the FP8 release is 128 GB of weights and needs a Mac Studio M3 Ultra with 256 GB. Apple measured it at FP8 with MLX: 4.95 tokens per second on an M3 Ultra and 7.70 on an M5 Ultra. A dense model reads every weight for every token, so it is slower than a mixture of experts model of similar size.
How much memory does Mistral Medium 3.5 need?
Weight size is parameter count × bits per weight; recommended memory adds 15% runtime headroom plus a KV-cache allowance for the 262,144-token context window.
| Quantization | Weights | Recommended RAM | Cheapest fit | Notes |
|---|---|---|---|---|
| Q4_K_M | ~77.6 GB | 104 GB | M4 Max (Mac Studio) | Best quality/size trade-off — the default for most deployments |
| Q8_0 | ~136 GB | 168 GB | M3 Ultra (Mac Studio) | Near-lossless; noticeably better on code and long reasoning |
| FP16 | ~256 GB | 304 GB | Thunderbolt 5 cluster | Full precision — research and eval baselines |
Which Mac should you pick?
Generation speed on Apple Silicon scales mostly with memory bandwidth, so the chip choice sets both whether the model fits and how fast it runs.
| Chip | Max RAM | Bandwidth | Fits Mistral Medium 3.5? | From |
|---|---|---|---|---|
| M4 · Mac mini | 32 GB | 120 GB/s | Cluster only | $99/mo |
| M4 Pro · Mac mini | 64 GB | 273 GB/s | Cluster only | $200/mo |
| M4 Max · Mac Studio | 128 GB | 546 GB/s | Q4 | $286/mo |
| M3 Ultra · Mac Studio | 256 GB | 819 GB/s | Q4 + Q8 | $571/mo |
Deploy Mistral Medium 3.5 in about 5 minutes
- 1
Build a Mac in the constructor — pick an M4 Max with at least 104 GB of unified memory.
- 2
Pick any Ollama-based stack (or a clean machine) and pull Mistral Medium 3.5 with one command — every deployment ships the OpenAI-compatible API.
- 3
Point your existing OpenAI SDK code at the new base_url and ship.
from openai import OpenAI
client = OpenAI(
api_key="mcy_live_...",
base_url="https://dep-xxxx.macyou.cloud/v1",
)
response = client.chat.completions.create(
model="mistral-medium-3-5",
messages=[{"role": "user", "content": "Hello!"}],
)License and what it's good at
Modified MIT License — Modified MIT: commercial and non commercial use, with exceptions for companies with large revenue; read the license file for the threshold. Made by Mistral AI.
- Dense model with a 256K context
- Text and image input
- Open weights that run on one Mac
Typical workloads: coding agents (mistral runs its vibe remote agents on it); long document work; private multimodal assistants.
Frequently asked questions
Q4 or FP8 on a Mac?
Q4_K_M, about 78 GB, fits a Mac with 128 GB. FP8 is 128 GB of weights and, according to Apple, up to 158 GB of memory with a 65,536 token context, so it needs an M3 Ultra with 256 GB.
How fast is Mistral Medium 3.5 on a Mac?
Apple measured 4.95 tokens per second on an M3 Ultra and 7.70 on an M5 Ultra at FP8 with MLX (Apple Machine Learning Research, 15 October 2026). Q4 reads about 60% of the bytes FP8 does for each token, so it generates faster on the same chip.