STORM+

Faster PyTorch optimizer

STORM+ is a fully adaptive stochastic gradient descent optimizer with momentum for nonconvex optimization, extending the STORM algorithm. It achieves optimal convergence rates without requiring knowledge of problem-specific parameters such as the Lipschitz constant or noise level, at the cost of two gradient evaluations per iteration.