Transformer Positional Encoding
Easy · Solved in PyTorch · Machine learning coding practice
Problem
In the Attention is All You Need paper, the authors use sinusoidal positional encodings to inject information about the relative or absolute position of tokens in a sequence.
Topics: transformer, torch, deep-learning, positional embeddings
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve Transformer Positional Encoding in the browser and run it against the tests.