Posts tagged with "MLSys"

Autotune, End to End — From Triton to FlyDSL

How kernel autotuning is designed in Triton, used in aiter / quack / CuteDSL, and consumed by inference engines like SGLang — and, from all of that, how to design a real autotune path for FlyDSL. With a full HTML deep dive carrying six hand-drawn SVG plates.

Long-Sequence MoE RL Training: From First Principles to MI300X

A first-principles read of Yan Bai's long-sequence MoE RL optimizations — Path B recompute, linear cross-entropy, FSDP2, chunked expert-parallel overlap — and what each one means on AMD MI300X / MI355X.

From Python to Silicon — A Compiler & Arch Primer for the Working ML Engineer

You can write production ML systems for years without knowing what IR, MLIR, LLVM, ISA, or FFI actually mean. This is the patch — a bilingual primer for the undergrad-CS-but-skipped-compilers crowd, with a full HTML deep dive carrying six hand-drawn SVG plates.

Attention Mechanisms — Full, Sparse, Linear, NSA & GLA

Breaking down Full, Sparse, and Linear Attention, all the way to DeepSeek NSA and Gated Linear Attention

TritonForge: Server-based Multi-turn RL for Triton Kernel Generation

End-to-end server-based RL training and evaluation system for Triton kernel generation across NVIDIA and AMD, built on slime + Megatron

Transformer Deep Dive (Math + Code)

Deconstructing Transformer's Self-Attention, LayerNorm, and MLP from math, code, and architecture perspectives