Skip to content
View Mattral's full-sized avatar
👀
I may be slow to respond.
👀
I may be slow to respond.

Highlights

  • Pro

Block or report Mattral

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
mattral/README.md

Mattral

ML Systems Engineer · Distributed Training · LLM Infrastructure · Multimodal Systems

I build systems that work at scale -- and try to understand why they fail when they do.


What I actually do

I work in the space between clean research ideas and the messy reality of clusters that fail, data that drifts, and models that need to stay honest in production.

Things I care about technically

  • Large-scale pre-training infrastructure -- MoE routing, fault-tolerant checkpointing, tensor/pipeline parallelism
  • LLM safety and observability -- keeping models (and the agents built on them) honest at inference time
  • The hardware-software boundary: SIMD, CUDA, kernel-level optimization
  • Novel architectures worth deploying, not just benchmarking

Things I care about less technically

  • Code that impresses interviewers but breaks on week two
  • Benchmarks that only win on synthetic data
  • Documentation that describes the happy path and nothing else

Upstream contributions

These are small but high-signal fixes in core infrastructure:

Project Status What was fixed Link
Triton Merged Fixed NaN handling in tl.argmin / tl.argmax so interpreter matches JIT behavior PR #10699 · write-up
TensorFlow Merged tf.experimental.numpy.swapaxes now raises a clear error on out-of-bounds axis instead of silent normalization or opaque XLA errors PR #122544 · write-up
Megatron-LM Merged Fixed crash in get_grad_norm_fp32 when gradient list is empty (common with frozen layers / tensor parallelism) PR #5530 · write-up

Selected work

I'd rather have a few things that are real than many that just look good on a profile.

Project What it is Where it actually stands
KANX Production-oriented Kolmogorov-Arnold Networks library (PyTorch + TensorFlow + ONNX) pip install kanx · Downloads · Colab · write-up · DOI · latest release v0.2.1
Composed-MoE-Engine MoE training runtime (research prototype): fused Triton router, 4D parallelism, expert-resharding recovery, async checkpointing 80.1× (T4) / 56.0× (A100) router-kernel forward speedup versus the CPU reference path (no GPU baseline yet) · CPU test suites run on every push (counts in the repo README) · three hardware-only bugs documented · validated on single GPUs (T4, A100) and at 2 ranks · multi-node and 8+ GPU runs open
filing-qa Question answering over financial filings with cited answers (hybrid dense+BM25 retrieval, numeric guardrail, AST-sandboxed calculator; chart extraction not yet tested) FastAPI + CLI · Helm chart included, not deployed · 119-question golden set · ~30% of questions miss at retrieval · calculation questions are the weakest
guardrail-rs Zero-Python LLM security reverse proxy in Rust (prompt injection, PII redaction, policy enforcement) Regex + PII pipeline adds 5–47 µs on 512 B–8 KB inputs (single local run, pipeline only) · ONNX stages experimental · injection detection measured on a public set (deepset, 263 prompts): pattern rules catch 1.5%, optional ML model 40%; PII detection not yet measured · fails open by default · five crates on crates.io · Colab · write-up
FlashSpec Speculative decoding for Hugging Face causal LMs with static-cache CUDA-graph fast path pip install flashspec · Downloads · v0.3.2 · 1.26–1.67× over compiled HF generate on one T4 (Qwen2.5-1.5B with 0.5B drafter, batch 1, 10 prompts per set) · lossless in float32 CPU tests; fp16 GPU greedy matched on 17 of 20 prompts · still Alpha · bandit draft selection experimental, did not beat the best fixed arm
simd-ml-kernels Hand-written AVX2/AVX-512/VNNI microkernels for core inference arithmetic x86-64 only · benchmarked against OpenBLAS, PyTorch and llama.cpp (single thread, mostly one laptop; per-case ratios in the repo) · pre-1.0 (v0.4.0) · educational reference, not a replacement for OpenBLAS, oneDNN or llama.cpp · limitations listed in the README

Also digging into

Not everything is a shipped tool -- some of it is just a question I wanted a real answer to.

  • Does fine-tuning break a transformer's copy mechanism? Write-up →

Other work

  • rlpost -- LLM post-training from scratch (SFT, reward model, PPO, DPO); parity-tested against TRL
  • production-vlm-engineering -- Reference scaffold for multimodal vision pipelines (config validation, batching, observability hooks); results are CPU/synthetic proxies
  • mcp-reliable -- Runtime observability for MCP servers (health, per-tool latency, output-shape drift)
  • PromptCanary -- Detects silent behavioral drift in LLM providers (pip install promptcanary)

Stack

Training & inference -- PyTorch TensorFlow Triton ONNX TensorRT FSDP2
LLM -- Transformers PEFT vLLM FastAPI
Infra -- NCCL Kubernetes Helm Ray
Observability -- Prometheus Grafana OpenTelemetry
Low-level -- C++ CUDA SIMD pybind11


Problem-solving

Algorithms are how I warm up. Systems are where I live.


A few honest notes

Work I do at my employer is not public. The repositories here are my own projects.

Repository READMEs are being updated to state current status and what has not been validated.

My path into this wasn’t linear. It started in mechatronics -- building systems with real sensors, actuators, and control loops that fail in ways no unit test catches -- and later working on ML systems inside industrial control and safety environments, where a silent failure has real consequences. That background is the root of the reliability focus you see in these projects.


Currently

  • Working on: the MoE engine’s chaos scenario A -- sudden node failure under expert resharding (recovers in roughly 85% of runs; failures attributed to a Gloo race in containers)
  • Reading: the Megatron-LM codebase and the FlexAttention paper
  • Thinking about: whether MFU tracking gives enough signal to catch silent training degradation early

🎶 Current frequency


Outside of work I’m usually reading something I don’t fully understand yet, listening to music that has no business being that good, and occasionally wondering if the model actually converged or if I just got lucky. I like working with people who say “I don’t know” without embarrassment and argue about architecture in good faith.

Pinned Loading

  1. KANX KANX Public

    Kolmogorov-Arnold Networks for TensorFlow and PyTorch, with ONNX export, a FastAPI service, a CLI, and Docker and Kubernetes configs.

    Python 38 8

  2. guardrail-rs guardrail-rs Public

    LLM security proxy written in Rust.

    Rust 14

  3. simd-ml-kernels simd-ml-kernels Public

    C++17 library of hand-written x86 SIMD kernels for the arithmetic at the core of neural-network inference: matrix multiplication, the matrix-vector product that dominates generating text with a lan…

    C++ 16 5

  4. Composed-Mixture-of-Experts-Engine Composed-Mixture-of-Experts-Engine Public

    Fault-tolerant MoE training runtime with fused Triton router, 4D parallelism, elastic recovery, and async checkpointing

    Python

  5. FlashSpec FlashSpec Public

    Lossless speculative decoding for HuggingFace causal language models, with a static-cache CUDA-graph fast path.

    Python 17

  6. filing-qa filing-qa Public

    Research prototype: multimodal RAG for financial filings. Hybrid dense+BM25 retrieval (RRF), numeric grounding checks, a sandboxed calculator and cited answers. Measured on a 119-question golden se…

    Python 91 21