Numerically Stable Softmax

Easy · Solved in Python · Machine learning coding practice

Problem

Implement the softmax function that converts a list of logits \mathbf{z} = [z_1, z_2, \dots, z_K]

into a probability distribution where each output is in (0, 1) and all outputs sum to 1.

Topics: softmax, numerical-stability, activation-function

The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve Numerically Stable Softmax in the browser and run it against the tests.

Related problems

• Vocabulary-Parallel Embedding and Cross-Entropy Hard
• Transformer Positional Encoding Easy
• Batch Normalization Easy
• Build Binary Classifier with Cross Entropy Loss Easy
• Binary Quantization and Bit Packing Easy
• Score Allowed Candidate Tokens Easy