← Back to the Lab
Notes on Local-First LLM Inference
Working notes on running capable language models on-device, and what it changes for product design.
Why this matters
Running models on-device reframes privacy, latency, and cost. These are ongoing notes — a living document rather than a finished argument.
Open questions
- Where is the quality/size break-even for genuinely useful on-device models?
- How should UX signal that inference is local versus remote?
- What does graceful degradation look like when a device cannot keep up?
Current take
Local-first is less about replacing the cloud and more about choosing, per interaction, where computation should happen.