Guides by Christian Munoz

Add a Local Ollama Model to a Tauri Desktop App

Get the Tauri Local-AI Starter Kit on Gumroad $199 one time

The problem

Calling a local model from a desktop app looks like one HTTP request, until you ship it. The model runtime may be missing, stopped or busy, the first answer can take minutes on a modest laptop, and a thoughtless setup lets any code in your project call any provider. Every one of those gaps ends up as a support email from a confused user.

The Tauri Local-AI Starter Kit already contains this plumbing, so you start from a working integration instead of a blank file. This guide follows the path of a request from the first run through to an error screen: installing Ollama, running the kit, choosing a model for the hardware, letting the user trigger the download and understanding the rules that keep the boundary tidy. It targets Windows only, since that is the one platform the kit is tested on.

Steps

  1. Install Ollama on your development computer

    Ollama is a free program from ollama.com that the end user also installs. Your app talks to it on http://127.0.0.1:11434 through its native /api/chat. Install it now so you can see the integration working rather than reading about it.

  2. Run the kit

    Unzip the kit, open PowerShell in the folder and run npm ci, npm run init-app, npm ci and npm run desktop:dev. The dev run opens the desktop app against a fresh temporary data folder, so experiments never touch real data.

  3. Walk through the first-run setup

    The setup screen recommends a model by total memory. Under 6 GB it suggests llama3.2:1b, from 6 GB to under 10 GB llama3.2:3b, from 10 GB to under 14.5 GB llama3.2:3b with an optional qwen2.5:7b, and above that qwen2.5:7b plus llama3.2:3b. The table lives in server/ai/recommend.js, so you can change the defaults.

  4. Let the user click Download model

    The Download model button asks the local Ollama to fetch the model. It runs only on a click, only for names listed in PULLABLE_MODELS, and can be cancelled. Never add a background or automatic download; the kit treats that as the one documented exception to its network rules, and it only works because it stays user-initiated.

  5. Choose tool and chat models

    In Settings > AI the user can pick models. If they have not, the app picks the largest installed model that supports tools (up to 14 billion parameters) for answers that read or change data, and a smaller text model for quick replies.

  6. Test the failure paths

    Stop Ollama and ask a question. Routes answer 503 with ai_not_configured or ollama_unavailable, and the UI shows Get Ollama or Check again. A full queue answers 429 ai_busy.

One door for every model call

All model calls go through server/ai/index.js, and a test in the kit fails if any other file imports a provider. That single boundary is where timeouts, queueing and consent are applied. Local Ollama is the default, a model on another computer needs the aiLan opt-in, and a cloud provider needs its own network opt-in plus consent.

Numbers are handled the same strict way: the code computes totals in SQL or JavaScript and the model only puts finished figures into words.

Slow hardware is a design input

On a computer without a graphics card the first answer can take a minute or two while the model loads. The kit waits up to 300 seconds for a first token, runs model calls one at a time and holds a queue of up to 8 more. Smaller models are quicker, which is why the 1B and 3B options sit at the low end of the memory table.

Pros and cons

Pros

  • The integration, queue, timeouts and error screens already exist, so you edit them instead of inventing them.
  • Model suggestions follow the user's memory, and downloads happen only on a click.
  • A single AI boundary enforced by a test keeps provider code out of the rest of the project.
  • The app works without Ollama installed, showing a clear next step instead of a raw error.

Cons

  • Your users must install a second program, Ollama, and download a model of a few gigabytes.
  • Answers on processor-only computers can be slow for the first request.
  • The kit is tested on Windows only, so you should not promise other systems.
  • The model list in recommend.js is a default; you need to evaluate models for your own users with the eval script.

Frequently asked questions

Does the kit include a model?

No. The user downloads one through Ollama by clicking Download model in the setup screen.

Can the app use Ollama on another computer?

Yes, but a non-loopback host needs the aiLan opt-in in Settings > Network. Without it the server answers 403 network_opt_in_required and sends nothing.

How do I check which models are good enough?

Run npm run eval:ai, which is a real-model check that needs Ollama and is not part of the normal test run.

Does the app break if Ollama is missing?

No. AI routes answer with a fixed message and the interface offers Get Ollama or Check again.

Get the Tauri Local-AI Starter Kit on Gumroad $199 one time