Run Llama 3.2 3B on Apple Silicon

Llama 3.2 3B is the fastest practical Llama: we measured 46.7 tokens/sec on a base M4 Mac mini with 16 GB (median of 3 runs, spread under 0.1). Weights are about 2 GB at Q4, so it runs on any Apple Silicon config with room to spare — the go-to model for high-throughput classification, routing, and agent glue logic.

How much memory does Llama 3B need?

Weight size is parameter count × bits per weight; recommended memory adds 15% runtime headroom plus a KV-cache allowance for the 8,192-token context window.

Which Mac should you pick?

Generation speed on Apple Silicon scales mostly with memory bandwidth, so the chip choice sets both whether the model fits and how fast it runs.

Deploy Llama 3B in about 5 minutes

1. 1

   Build a Mac in the constructor — pick an M4 with at least 8 GB of unified memory.

2. 2

   Pick any Ollama-based stack (or a clean machine) and pull Llama 3.2 3B with one command — every deployment ships the OpenAI-compatible API.

3. 3

   Point your existing OpenAI SDK code at the new base_url and ship.

from openai import OpenAI

client = OpenAI(
    api_key="mcy_live_...",
    base_url="https://dep-xxxx.macyou.cloud/v1",
)
response = client.chat.completions.create(
    model="llama-3-2-3b",
    messages=[{"role": "user", "content": "Hello!"}],
)

License and what it's good at

Llama 3.2 Community License — Free for commercial use below 700M monthly active users; attribution required. Made by Meta.

• 46.7 tok/s measured on the cheapest config — real-time by a wide margin
• Leaves 10+ GB free on a 16 GB machine for your app or a second model
• Surprisingly capable at summarization, extraction, and routing

Typical workloads: high-volume classification and tagging; router/planner model in agent pipelines; autocomplete and draft generation; guardrails and output validation — cheap second opinions.

Frequently asked questions

Related

• Run Llama 3.1 8B on Apple Silicon →
• Run Qwen 2.5 7B on Apple Silicon →
• Deploy Llama 3.2 on a Mac Mini M4 Pro in Under 2 Minutes →
• Buying vs renting a Mac for AI →