AdamW Optimizer
Medium · Solved in PyTorch · Machine learning coding practice
Problem
In this challenge, you will implement AdamW (Adam with decoupled Weight decay) (Loshchilov & Hutter, 2017), the de facto optimizer for training modern large-scale models like GPT and BERT.
Topics: optimization
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve AdamW Optimizer in the browser and run it against the tests.