Ollama 0.9: concurrent model serving, multi-GPU split, and REST API v2 for local AI

Think of Ollama as a kitchen where, until now, only one dish could be cooked at a time. Before you could start a new recipe, the chef had to completely clear the counter, put everything away, and start fresh. Ollama 0.9 turns that kitchen into a proper multi-station setup: several dishes can be in progress simultaneously, each on its own burner, ready to serve when needed.

In practical terms, your local machine can now keep multiple AI models loaded in memory at the same time. If you are using one model to help write code and another to answer questions in a different language, you no longer have to wait for the first one to fully unload before the second one becomes available.

The new caching feature is equally useful. Instead of the AI rereading the entire conversation from scratch every time you send a message, it remembers where it left off. This makes responses noticeably faster in back-and-forth conversations.

For people with multiple graphics cards in their machine (common in workstations and small servers), Ollama 0.9 can now spread the work across all of them automatically. It is like having two workers carry a heavy load together instead of one person struggling alone.

The Windows installer is also easier to use, and the Docker image takes up significantly less disk space than before. Together, these improvements make Ollama 0.9 a meaningful milestone for anyone running capable AI models locally without relying on cloud services.