xzo

A language model can only read so much text at once. In a long conversation, earlier messages may no longer fit. xzo runs between your app and a local model, saves older messages to disk, and retrieves relevant passages for later questions.

To use it, point an app that supports the OpenAI chat-completions API at xzo instead of directly at the model. xzo summarizes older messages in the background. When the conversation exceeds the context limit, it keeps your instructions and recent messages, replaces older messages with a recap, and adds stored passages that match the question. For “how many” and “list all” questions, it reads all the stored passages.

xzo is written in Rust and works with llama.cpp, Ollama, MLX, and LM Studio. It does not need Python or a separate GPU. Results so far come from tests with Qwen3.5-9B; other models have not yet been measured.

Features

• Connects through the OpenAI chat-completions API
• Handles tool calls and results. Long tool outputs become previews, with a note explaining how to request the rest
• Runs locally and stores each conversation separately
• Preserves your instructions and latest message, and tracks summaries across restarts to avoid summarizing the same text twice
• Labels retrieved text as reference notes rather than instructions
• In tests with Qwen3.5-9B and an 8,192-token context window, 10 of 10 agent sessions of about 225 turns answered correctly, compared with 2 of 10 without xzo

What we did

We designed and built xzo, and released it under the Apache-2.0 license.

Related links and products

• GitHub ↗ — Source, quick start, settings, and the full test results.