Skip to content
wispoffatesPublic
forked from ggml-org/llama.cpp

About

llama.cpp tuned for NVIDIA Tesla P100 (Pascal, sm_60) GPUs.

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

 
 

Latest commit

 

History

12,818 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Note

This is p100-series: llama.cpp tuned for NVIDIA Tesla P100 (Pascal, sm_60) GPUs. It is upstream llama.cpp plus a linear stack of Pascal patches: faster decode matvecs, two-GPU tensor split with an in-kernel P2P AllReduce, an MoE expert cache for models larger than VRAM, and Qwen 3.8 Flash-Next support. The stack is replayed onto new upstream commits and gated on real P100s (correctness, KL divergence, speed, serving) before each p100-series-vX.Y tag. Other GPUs keep upstream's code paths.

Current release: p100-series-v5.44 (on upstream 41d8297fe, 2026-11-19).

Who wrote the 247 code patches in p100-series-v5.44 (git authorship; carried commits link their originals):

Author Patches What
wispoffates (maintainer, with Claude Code) 200 Pascal kernels, two-GPU tensor-split engine, expert cache, Flash-Next, checkpoint staging; 5 of them also credit qr0n, buun or Keoian as co-author / source
Kaden (Kmic-68) 33 the original sm_60 matvec rebuild, fp16 Q6_K verify matvecs, cuBLAS ALGO6, sampler prefilter, fp32 fold of the tile-FA accumulator
drrros 5 pipeline parallelism with MoE experts in host memory (upstream PR #29963)
Keoian (via Kmic-68) 4 faster repetition penalties, LoRA under tensor split, cheaper chat-turn checkpoints
Kane 2 gated-delta-net decode chain fused into 4 kernels, IQ matvec tuning for sm_60 (their per-expert MoE MMQ tile, v5.37-v5.40, is upstream since #31047)
lich-77 2 the GPU-resident MoE expert LRU cache our expert cache builds on
wrj97 1 draft KV kept out of context checkpoints (upstream PR #29509)

Everything below this note is upstream's README.

llama.cpp

llama.cpp

Description

The main goal of llama.cpp is to enable LLM (and MLLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.

The llama.cpp project is built on top of the ggml library.

Quick start

A few options to get llama.cpp installed on your machine:

# curl
curl -LsSf https://llama.app/install.sh | sh

# powershell
irm https://llama.app/install.ps1 | iex

Once installed:

# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
VLM session with `llama cli` VLM session with llama cli Llama Chat, the built-in chat app, against `llama serve` running Qwen 3.6 Llama Chat, the built-in chat app, against llama serve

Documentation

Tools

Development

Contributing

  • Contributors can open PRs
  • Collaborators will be invited based on contributions
  • Maintainers can push to branches in the llama.cpp repo and merge PRs into the master branch
  • Any help with managing issues, PRs and projects is very appreciated!
  • Read the CONTRIBUTING.md for more information

Acknowledgements

  • yhirose/cpp-httplib - Single-header HTTP server, used by llama-server - MIT license
  • nothings/stb - Single-header image format decoder, used by multimodal subsystem - Public domain
  • nlohmann/json - Single-header JSON library, used by various tools/examples - MIT License
  • mackron/miniaudio - Single-header audio format decoder, used by multimodal subsystem - Public domain
  • sheredom/subprocess.h - Single-header process launching solution for C and C++ - Public domain

About

llama.cpp tuned for NVIDIA Tesla P100 (Pascal, sm_60) GPUs.

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages