Marcus Andrey Study
Ler em português Library

AI for UX · Labs · 6 min read

Run a model on your laptop

A hands-on lab: run an open language model on your own computer, with an app or a few terminal commands. What you need, which small models to try, what stays on your device, what we measured on a Mac, and how to judge the answers.

Four test tubes, one for each lab: run a model on your laptop, make images locally, write a skill, write an eval set. Point at a tube and its experiment starts to bubble. Illustration: Marcus Andrey, drawn with Hairline (MIT, Lucas Marques).

The fastest way to understand a language model is to run one yourself. On a recent laptop, a small open model answers in seconds, works without internet, and nothing you type leaves the machine. It's also humbling: small models are clearly weaker than the big cloud assistants, and seeing how they fail teaches you more about AI than any article. This lab takes about thirty minutes, most of it downloading.

What you'll be able to do after this lab:

  • Install a local model runner, with an app or in the terminal
  • Choose a small model that fits your computer
  • Explain what stays on your device and what doesn't
  • Compare a local model with a cloud assistant on a real design task

Products and versions are as of November 2026. They change fast; check each link before you start.


1. Why run a model locally

LocalPrivacy

What you type stays on the computer, with a local model.

Cloud: it goes to the vendor, under their terms
LocalOffline and free

Works without internet; no per-use cost.

Cloud: needs a connection and a plan
CloudQuality

Large cloud models are clearly stronger.

Local: small models make more mistakes
CloudEffort

Nothing to install or update.

Local: you download and manage models
Local and cloud models trade privacy and cost for quality and convenience. Many teams use both. Diagram: Marcus Andrey, CC BY 4.0.
Empty, numbered glass test tubes standing in a steel rack.
Clean tubes, ready for an experiment. A lab is where you try things on purpose, write down what happened, and learn more from what failed than from what worked. Photo: J.N. Eskra, Wikimedia Commons, CC BY-SA 4.0.

2. What you need

  • A computer from the last few years. For Apple, a Mac with an M-series chip; on Windows, a recent PC (a graphics card helps but isn't required for small models).
  • Memory (RAM): with 8 GB, use models of about 3 to 4 billion parameters; with 16 GB or more, you can try larger ones. LM Studio publishes this guidance; Ollama no longer does.
  • Disk space: 2 to 10 GB per model.
  • Thirty minutes, mostly for the download.

3. Three ways to start

Tool How you use it Good for As of Nov 2026
Ollama A chat app for Mac and Windows, plus terminal commands The easiest start; drag in a PDF and ask about it Free, open source; Mac needs macOS 14 or later
LM Studio A desktop app with a model catalogue Browsing and comparing models without a terminal Free, including for work inside your company ("internal business purposes"); Apple silicon Macs on macOS 14 or later
mlx-lm Terminal commands, Apple silicon only Speed on a Mac, and the next step: fine-tuning Free, MIT licence; needs Python 3.11 or later

The no-terminal path: Ollama's app

  1. Download Ollama from ollama.com/download and open it.
  2. In the chat window, pick a model from the list (for example qwen3.5:4b). It downloads the first time.
  3. Type a question. Turn off Wi-Fi and ask again: it still works.

The terminal path

# Ollama, after installing the app
ollama run qwen3.5:4b

# or mlx-lm on a Mac with Apple silicon
pip install mlx-lm
mlx_lm.chat --model mlx-community/Qwen3.5-4B-MLX-4bit

4. Which model to try

Model Download Licence Fits in
Qwen3.5 4B (qwen3.5:4b) about 3.3 to 4 GB Apache 2.0 8 GB RAM
Llama 3.2 3B (llama3.2:3b) about 2 GB Llama Community License: free, with attribution ("Built with Llama") and limits for very large companies 8 GB RAM
Gemma 4 E4B (gemma4:e4b) about 6.6 to 9.5 GB Apache 2.0 16 GB RAM

Sizes come from each model's page on ollama.com/library. "Fits in" is a rule of thumb.


5. What we measured

We ran Qwen3.5 4B (4-bit) with mlx-lm 0.32 on a MacBook Pro with an M3 Max chip and 36 GB of memory, offline, and asked:

"Write three short, friendly empty-state messages for a design team's shared asset library that has no files yet. One line each."

Thinking on (default) Thinking off
Tokens before the answer appeared 200, all spent "thinking", and no answer yet 0
Answer length (cut off) 56 tokens
Speed about 95 tokens per second about 96 tokens per second
Memory used 2.6 GB 2.6 GB
Download 3.0 GB 3.0 GB

The answer, with thinking off:

1. Your design vault is empty—let's build something amazing together!
2. No files yet? This is the perfect blank canvas for your next big idea.
3. Ready to drop in your first asset? We can't wait to see what you create!

What this shows:

  • Reasoning costs tokens. Many new models "think" first by default. With a 200-token limit, all of it went to thinking. In mlx-lm you can turn it off with --chat-template-config '{"enable_thinking": false}'.
  • Fast and private isn't the same as good. The lines are generic and over-excited ("amazing", "big idea"), and none tells the person what to do. A content designer would rewrite all three. See AI for content design.

6. What stays on your device

  • Stays: your prompts, the answers and any files you drop in, if you use a local model. Ollama says: "We don't see your prompts or data when you run locally." LM Studio says local chats are never sent off the device.
  • Doesn't: the model download itself comes from the internet; cloud models and web search features send data out; app updates and anonymous usage statistics may too. Read the app's privacy page.
  • Your organisation's rules still apply. Local doesn't mean approved: ask before you use work data. See AI governance and data protection.

7. Common mistakes

  • Choosing a model too big for your memory: the computer slows to a crawl.
  • Assuming "local" when the model is a cloud one.
  • Trusting facts from a small model. They make things up more often than large ones. See Hallucination.
  • Downloading models from unknown sources. Use the official libraries.

8. Vocabulary

Term Meaning
Open model A model whose weights you can download and run yourself
Parameters The numbers a model learned; "4B" means about 4 billion
Quantised (4-bit) Compressed so it needs less memory, with a small loss of quality
Tokens per second How fast the model writes
Reasoning ("thinking") Text the model writes for itself before it answers
Context window How much text the model can consider at once

9. Try it

  1. Install Ollama or LM Studio and download one small model.
  2. Turn off Wi-Fi. Ask it for three empty-state messages for a product you know.
  3. Ask a cloud assistant the same thing.
  4. Grade both with the same three criteria: clear, on-voice, tells the person what to do next.

Reflection questions: Which tasks in your work would you trust to a small local model? Which would you never? What did the local model get wrong that the cloud one didn't?

Checklist

Learn more

Books, courses and videos to go deeper

Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.

How a model learns: train one on your laptop · What AI is (and isn't) · AI tools landscape for designers · AI for content design · AI governance and data protection · Hallucination · Context window

Saved only on this device.

↑ ↓ to move · Enter to open · Esc to close