Implementing Leaky ReLU with Manual Autograd
Easy · Solved in PyTorch · Machine learning coding practice
Problem
Standard ReLU has a "dying ReLU" problem where neurons can become inactive and only output zero. Leaky ReLU fixes this by allowing a small, non-zero gradient when the input is negative.
Topics: pytorch, autograd, deep-learning, activation-functions
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve Implementing Leaky ReLU with Manual Autograd in the browser and run it against the tests.