Run Mistral Small 4 on Apple Silicon
Mistral Small 4 is a 119 billion parameter mixture of experts model with 6.5B active per token, a 256K context and image input, released by Mistral AI in March 2026 under Apache 2.0. At Q4_K_M the weights are about 72 GB, so a Mac Studio M4 Max with 128 GB runs it on one machine. Only 6.5B parameters are read for each token, so generation is much faster than a dense model of the same size.
How much memory does Mistral Small 4 need?
Weight size is parameter count × bits per weight; recommended memory adds 15% runtime headroom plus a KV-cache allowance for the 262,144-token context window.
| Quantization | Weights | Recommended RAM | Cheapest fit | Notes |
|---|---|---|---|---|
| Q4_K_M | ~72.1 GB | 96 GB | M4 Max (Mac Studio) | Best quality/size trade-off — the default for most deployments |
| Q8_0 | ~126.4 GB | 152 GB | M3 Ultra (Mac Studio) | Near-lossless; noticeably better on code and long reasoning |
| FP16 | ~238 GB | 280 GB | Thunderbolt 5 cluster | Full precision — research and eval baselines |
Which Mac should you pick?
Generation speed on Apple Silicon scales mostly with memory bandwidth, so the chip choice sets both whether the model fits and how fast it runs.
| Chip | Max RAM | Bandwidth | Fits Mistral Small 4? | From |
|---|---|---|---|---|
| M4 · Mac mini | 32 GB | 120 GB/s | Cluster only | $99/mo |
| M4 Pro · Mac mini | 64 GB | 273 GB/s | Cluster only | $200/mo |
| M4 Max · Mac Studio | 128 GB | 546 GB/s | Q4 | $286/mo |
| M3 Ultra · Mac Studio | 256 GB | 819 GB/s | Q4 + Q8 | $571/mo |
Deploy Mistral Small 4 in about 5 minutes
- 1
Build a Mac in the constructor — pick an M4 Max with at least 96 GB of unified memory.
- 2
Pick any Ollama-based stack (or a clean machine) and pull Mistral Small 4 with one command — every deployment ships the OpenAI-compatible API.
- 3
Point your existing OpenAI SDK code at the new base_url and ship.
from openai import OpenAI
client = OpenAI(
api_key="mcy_live_...",
base_url="https://dep-xxxx.macyou.cloud/v1",
)
response = client.chat.completions.create(
model="mistral-small-4",
messages=[{"role": "user", "content": "Hello!"}],
)License and what it's good at
Apache 2.0 — Apache 2.0: commercial use is allowed under the standard license terms. Made by Mistral AI.
- Apache 2.0 license
- Text and image input
- Fits one 128 GB Mac at Q4
Typical workloads: agents and tool calling on a single machine; reading documents and screenshots; replacing hosted mid size models in private pipelines.
Frequently asked questions
Can a 64 GB Mac run Mistral Small 4?
Not at Q4_K_M: the weights alone are about 72 GB. A Mac with 128 GB of unified memory runs it with room for context.
Does it understand images on a Mac?
The model takes text and image input and answers in text. Image support depends on the runtime you use, so check that your version of llama.cpp, MLX or Ollama lists it for this model.