Decode with TopP Sampling
Medium · Solved in PyTorch · Machine learning coding practice
Problem
Top-P Sampling, also called nucleus sampling, is the decoding strategy introduced in The Curious Case of Neural Text Degeneration (Holtzman et al., 2019). Where top-k sampling keeps a fixed number of candidates, top-p keeps a fixed amount of probability mass, so the candidate set shrinks when the model is confident and grows when it is not.
Topics: inference, sampling, autoregressive
The full statement, worked examples, hints and the test suite are available once you sign in. You can then solve Decode with TopP Sampling in the browser and run it against the tests.