Gated DeltaNet: Q/K L2 Normalization
Easy · Solved in PyTorch · Machine learning coding practice
Problem
Normalize each query and key head over its feature axis with a stable L2 denominator.
Topics: gated-deltanet, linear-attention, llm-internals
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve Gated DeltaNet: Q/K L2 Normalization in the browser and run it against the tests.