LocalAI: OpenAI API drop-in replacement with local models
In one sentence LocalAI, the Go REST server by Ettore Di Giacinto (mudler) compatible with the OpenAI API, reaches version 1.9.0 on May 11, 2023: besides ggml text models it handles audio transcription with whisper.cpp and embeddings. Apps written for OpenAI can point to a local server.
Many applications are written to use OpenAI's APIs. LocalAI takes advantage of that: it provides a server that speaks the same language as the OpenAI APIs, but processes requests locally instead of sending data to the cloud.
The project started in March 2023 as llama-cli, a command-line interface for llama.cpp, and shortly afterwards was renamed LocalAI to focus on the OpenAI-compatible API. It runs on consumer-grade CPUs and ships as a container image.
In early May embeddings and audio transcription were added; version 1.9.0 on May 11 brought important whisper updates and support for more GPT4All model families. For a company that wants to use AI on internal documents without them leaving the network, only the address the application connects to changes.
Companies
mudler (Ettore Di Giacinto)
Tools
LocalAI, llama.cpp, whisper.cpp, GPT4All
Tags
Sources