Things we built
without a client.
Kernel work and benchmarks we run on our own time, with the same tools and the same rigour we bring to your cluster.
PROJECTS
FlashAttention-3, variable-length.
Profiling and optimising variable-length attention kernels on NVIDIA Hopper, aiming for an upstream contribution to FlashAttention-3.
Reduction, softmax and GEMM, by hand.
Hand-optimised kernels, profiled with Nsight Compute and checked against the roofline at every step. Every version is benchmarked against the last.
BENCHMARK · H100
REDUCTION
3.1 TB/s · 92% OF PEAK BW
SOFTMAX
2.9 TB/s · 87% OF PEAK BW
GEMM, FP16
91% OF cuBLAS
FA3 VARLEN FWD
1.3× VS BASELINE