AI for UX · Labs · 6 min read
Run a model on your laptop
A hands-on lab: run an open language model on your own computer, with an app or a few terminal commands. What you need, which small models to try, what stays on your device, what we measured on a Mac, and how to judge the answers.
The fastest way to understand a language model is to run one yourself. On a recent laptop, a small open model answers in seconds, works without internet, and nothing you type leaves the machine. It's also humbling: small models are clearly weaker than the big cloud assistants, and seeing how they fail teaches you more about AI than any article. This lab takes about thirty minutes, most of it downloading.
What you'll be able to do after this lab:
- Install a local model runner, with an app or in the terminal
- Choose a small model that fits your computer
- Explain what stays on your device and what doesn't
- Compare a local model with a cloud assistant on a real design task
Products and versions are as of November 2026. They change fast; check each link before you start.
1. Why run a model locally
What you type stays on the computer, with a local model.
Cloud: it goes to the vendor, under their termsWorks without internet; no per-use cost.
Cloud: needs a connection and a planLarge cloud models are clearly stronger.
Local: small models make more mistakesNothing to install or update.
Local: you download and manage models
2. What you need
- A computer from the last few years. For Apple, a Mac with an M-series chip; on Windows, a recent PC (a graphics card helps but isn't required for small models).
- Memory (RAM): with 8 GB, use models of about 3 to 4 billion parameters; with 16 GB or more, you can try larger ones. LM Studio publishes this guidance; Ollama no longer does.
- Disk space: 2 to 10 GB per model.
- Thirty minutes, mostly for the download.
3. Three ways to start
| Tool | How you use it | Good for | As of Nov 2026 |
|---|---|---|---|
| Ollama | A chat app for Mac and Windows, plus terminal commands | The easiest start; drag in a PDF and ask about it | Free, open source; Mac needs macOS 14 or later |
| LM Studio | A desktop app with a model catalogue | Browsing and comparing models without a terminal | Free, including for work inside your company ("internal business purposes"); Apple silicon Macs on macOS 14 or later |
| mlx-lm | Terminal commands, Apple silicon only | Speed on a Mac, and the next step: fine-tuning | Free, MIT licence; needs Python 3.11 or later |
The no-terminal path: Ollama's app
- Download Ollama from ollama.com/download and open it.
- In the chat window, pick a model from the list (for example
qwen3.5:4b). It downloads the first time. - Type a question. Turn off Wi-Fi and ask again: it still works.
The terminal path
# Ollama, after installing the app
ollama run qwen3.5:4b
# or mlx-lm on a Mac with Apple silicon
pip install mlx-lm
mlx_lm.chat --model mlx-community/Qwen3.5-4B-MLX-4bit
4. Which model to try
| Model | Download | Licence | Fits in |
|---|---|---|---|
Qwen3.5 4B (qwen3.5:4b) |
about 3.3 to 4 GB | Apache 2.0 | 8 GB RAM |
Llama 3.2 3B (llama3.2:3b) |
about 2 GB | Llama Community License: free, with attribution ("Built with Llama") and limits for very large companies | 8 GB RAM |
Gemma 4 E4B (gemma4:e4b) |
about 6.6 to 9.5 GB | Apache 2.0 | 16 GB RAM |
Sizes come from each model's page on ollama.com/library. "Fits in" is a rule of thumb.
5. What we measured
We ran Qwen3.5 4B (4-bit) with mlx-lm 0.32 on a MacBook Pro with an M3 Max chip and 36 GB of memory, offline, and asked:
"Write three short, friendly empty-state messages for a design team's shared asset library that has no files yet. One line each."
| Thinking on (default) | Thinking off | |
|---|---|---|
| Tokens before the answer appeared | 200, all spent "thinking", and no answer yet | 0 |
| Answer length | (cut off) | 56 tokens |
| Speed | about 95 tokens per second | about 96 tokens per second |
| Memory used | 2.6 GB | 2.6 GB |
| Download | 3.0 GB | 3.0 GB |
The answer, with thinking off:
1. Your design vault is empty—let's build something amazing together!
2. No files yet? This is the perfect blank canvas for your next big idea.
3. Ready to drop in your first asset? We can't wait to see what you create!
What this shows:
- Reasoning costs tokens. Many new models "think" first by default. With a 200-token limit, all of it went to thinking. In mlx-lm you can turn it off with
--chat-template-config '{"enable_thinking": false}'. - Fast and private isn't the same as good. The lines are generic and over-excited ("amazing", "big idea"), and none tells the person what to do. A content designer would rewrite all three. See AI for content design.
6. What stays on your device
- Stays: your prompts, the answers and any files you drop in, if you use a local model. Ollama says: "We don't see your prompts or data when you run locally." LM Studio says local chats are never sent off the device.
- Doesn't: the model download itself comes from the internet; cloud models and web search features send data out; app updates and anonymous usage statistics may too. Read the app's privacy page.
- Your organisation's rules still apply. Local doesn't mean approved: ask before you use work data. See AI governance and data protection.
7. Common mistakes
- Choosing a model too big for your memory: the computer slows to a crawl.
- Assuming "local" when the model is a cloud one.
- Trusting facts from a small model. They make things up more often than large ones. See Hallucination.
- Downloading models from unknown sources. Use the official libraries.
8. Vocabulary
| Term | Meaning |
|---|---|
| Open model | A model whose weights you can download and run yourself |
| Parameters | The numbers a model learned; "4B" means about 4 billion |
| Quantised (4-bit) | Compressed so it needs less memory, with a small loss of quality |
| Tokens per second | How fast the model writes |
| Reasoning ("thinking") | Text the model writes for itself before it answers |
| Context window | How much text the model can consider at once |
9. Try it
- Install Ollama or LM Studio and download one small model.
- Turn off Wi-Fi. Ask it for three empty-state messages for a product you know.
- Ask a cloud assistant the same thing.
- Grade both with the same three criteria: clear, on-voice, tells the person what to do next.
Reflection questions: Which tasks in your work would you trust to a small local model? Which would you never? What did the local model get wrong that the cloud one didn't?
Checklist
Learn more
- Tools: Ollama · Ollama docs · LM Studio · mlx-lm
- Models: Ollama library · Qwen on Hugging Face
Books, courses and videos to go deeper
- Hands-On Large Language ModelsJay Alammar and Maarten GrootendorstBook · O'Reilly, 2024 (Amazon)
- AI EngineeringChip HuyenBook · O'Reilly, 2025
Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.
Related
How a model learns: train one on your laptop · What AI is (and isn't) · AI tools landscape for designers · AI for content design · AI governance and data protection · Hallucination · Context window