Skip to content
#

single-gpu

Here are 38 public repositories matching this topic...

Run Kimi K3.1 Pro and DeepSeek V4.1 Flash 1017 locally with Argodrive DS4: NVIDIA CUDA or Apple silicon Metal, NVMe expert streaming, caches and weight preparation. Setup and measured RTX PRO 6000 / M5 Max results included. Based on antirez/ds4; no separate Argodrive app required.

  • Updated Nov 19, 2026
  • C

Run Qwen3.8-Flash-Next on ONE RTX 3090 (24 GB) + 64 GB RAM: 128K context, up to 2,100 tok/s prefill, 43–51 tok/s decode. vLLM runtime with hot MoE experts on the GPU and cold experts computed on the CPU, INT8 KV cache, Docker, OpenAI-compatible API.

  • Updated Sep 30, 2026
  • Python

Engine for two MoE models only - DeepSeek-V4.1-Flash (main) and GLM-5.3-Flash - on one 96 GB GPU, experts offloaded to CPU RAM, 262K context. Pieced together from what we had (DDR4, PCIe 4); DDR5 would do better. Sleeps/wakes in seconds to share the GPU. OpenAI/Anthropic API, works behind LiteLLM.

  • Updated Nov 16, 2026
  • C++

Add this topic to your repo

To associate your repository with the single-gpu topic, visit your repo's landing page and select "manage topics."

Learn more