↓Skip to main content
Nick Sarkauskas

Nick Sarkauskas

Senior Software Engineer at NVIDIA

Recent

Multihead attention. Operations are purple, given tensors are green. Each head is its own box.

Notes on Transformers

·3185 words·15 mins
Walking through the shapes of every operation in GPT-2 small, with some extras about inference