Fastest LLM inference on Apple Silicon — now on NVIDIA GB10 (DGX Spark) and AMD Strix Halo / Strix Point, too.
Pull a model from HuggingFace, chat with it, or serve it over OpenAI-, Anthropic- and Ollama-compatible APIs — all from one CLI.
BaseRT runs large language models locally on Apple Silicon (Metal), NVIDIA
GB10 / DGX Spark (CUDA) and AMD Strix Halo / Strix Point (ROCm), accelerated by
hand-written GPU kernels. This repository is the open ecosystem around the
engine: the basert CLI (model hub + converter), the .base model format, the
public C API, and language bindings for Python, Node, Rust, and Swift.
- One CLI for everything —
basert pull,chat,serve,convert. - A local API server —
basert serveruns Superfluid: OpenAI-, Anthropic- and Ollama-compatible endpoints, tool calls, continuous batching, prefix caching and speculative decoding. - Multimodal — text, vision, and audio on supported models.
- Its own
.baseformat — affine quantization (Q2–Q8), optional AWQ calibration, and signed bundles. - Bindings everywhere — Python, Node, Rust, and Swift over a stable C API.
- A drop-in coding-agent backend — see below.
Documentation: basert.basecompute.co (source in
docs/). The server's API is documented at docs.basecompute.co.
One of:
- Apple Silicon (M1 or later), macOS 15+.
- NVIDIA GB10 (DGX Spark), Linux / arm64 with a CUDA driver.
- AMD Strix Halo (Ryzen AI Max, gfx1151) or Strix Point (Ryzen AI 300,
gfx1150), Linux / x86-64 on Ubuntu 24.04 or newer, with the ROCm 10 runtime
from AMD's apt repository (below). The kernel must be one AMD supports for
RDNA3.5: on Ubuntu 24.04 the HWE kernel
6.17.0-19.19~24.04.2or later or the OEM kernel6.14.0-1018or later, Ubuntu 26.04's own kernel, and elsewhere Linux 6.18.4 or later (AMD's list).
On AMD, add AMD's ROCm 10 repository, then install the runtime with the BLAS
kernels for your chip: amdrocm-blas10.0-gfx1151 on Strix Halo,
amdrocm-blas10.0-gfx1150 on Strix Point. AMD serves the repository for
Ubuntu 24.04 (ubuntu2404, as below) and 26.04 (ubuntu2604). The installer
checks for all of this and prints the commands for your machine.
sudo curl -fsSL https://stable.repo.amd.com/rocm/gpg/packages.gpg -o /etc/apt/keyrings/amdrocm.asc
echo 'deb [arch=amd64 signed-by=/etc/apt/keyrings/amdrocm.asc] https://stable.repo.amd.com/rocm/core/packages/ubuntu2404 stable main' | sudo tee /etc/apt/sources.list.d/amdrocm.list
sudo apt update
sudo apt install amdrocm-blas10.0-gfx1151
sudo usermod -aG render "$USER" # access to the GPU; log in again afterwardsFrom 0.5.3 the engine also uses the XDNA2 NPU beside the GPU to read prompts
faster, when the machine has the NPU's driver (/dev/accel/accel0), AMD's XRT
(sudo apt install libxrt2 libxrt-npu2 on Ubuntu 26.04) and an unlimited
locked-memory limit. Without them it runs on the GPU alone; the installer says
which are missing (details).
curl -LsSf https://basecompute.co/install.sh | shThis installs the prebuilt engine for your platform and the basert CLI to
~/.basert and adds it to your PATH. Restart your shell afterward (or
export PATH="$HOME/.basert:$PATH"). Update later with basert self update.
Or use a package manager:
brew install basecompute/tap/basert
pip install basert # also pipx / uv tool
npm install -g @basecompute/basertSee Installation for the details and for building from source.
basert pull Qwen/Qwen3-4B # download from HuggingFace + convert to .base
basert list # show installed modelsModels are cached under ~/Library/Caches/baseRT/models on macOS and
~/.cache/baseRT/models on Linux (override with $BASERT_MODELS_DIR). To pick a specific quant build, append :default-q8 to
the id (or pass --variant default-q8).
basert chat Qwen/Qwen3-4Bbasert serve Qwen/Qwen3-4B --api-key "$(uuidgen)" --port 8080curl http://127.0.0.1:8080/v1/chat/completions \
-H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen3-4B","messages":[{"role":"user","content":"Hello!"}]}'import baseRT
model = baseRT.Model("models/your-model.base")
print(model.generate_text("The capital of France is", max_tokens=64))Node, Rust, and Swift bindings work the same way — see Bindings.
BaseRT runs as a local backend for the pi coding agent through the pi-basert extension, which auto-discovers whatever models your server is running.
# 1. Serve a model (a dense model is best for tool calling)
basert serve basecompute/gemma-4-E2B-it
# 2. Install the extension and launch pi
pi install git:github.com/basecompute/pi-basert
piInside pi, run /model to pick a served model. See the
pi-basert README for details.
basert is a single front-end. Model-management commands run natively; chat,
complete and bench run the matching basert-<cmd> engine binary, and serve
runs Superfluid, installing it on first use.
| Command | What it does |
|---|---|
basert pull <id> |
Download from HuggingFace (convert-on-pull) or the catalog |
basert list |
List installed models (--remote to include the catalog) |
basert convert <src> |
Convert a local GGUF / HF / MLX checkpoint to .base |
basert chat <model> |
Interactive chat |
basert serve <model> |
HTTP server (Superfluid; --model repeats for multi-model) |
basert complete <model> |
One-shot completion (text / image / audio) |
basert bench <model> |
Throughput benchmark |
basert inspect <model> |
Dump a .base header + tensor inventory |
basert sign / verify / keygen |
ed25519 signing of .base bundles |
basert computearena |
Run a ComputeArena benchmark and save a signed report |
basert superfluid install / status |
Install or check the server basert serve runs |
basert self update / check / list / use |
Update basert, or switch between installed versions |
Full flag reference: CLI reference.
| Path | Contents |
|---|---|
base-convert/ |
The basert CLI (Rust): model hub, converter, launcher — plus the .base format and quantization specs and the generic quant profiles. |
include/baseRT/ |
The stable public C API (baseRT.h, types.h). |
bindings/ |
Python, Node, Rust, and Swift. |
benchmarks/ |
Scripts and reference results. |
docs/ |
The documentation site (basert.basecompute.co). |
This repository — the CLI, .base format, public headers, and bindings — is
licensed under Apache-2.0. The engine ships as a separate prebuilt binary
under its own license; see
Engine releases.
Apache-2.0 — see LICENSE and NOTICE. The prebuilt engine
binary distributed via Releases
is proprietary and ships under its own license.