awesome-soms
github.com/cor3bit/awesome-soms ↗A curated list of resources for second-order stochastic optimization
4
GitHub Stars
32
Curated Resources
3
Categories
19 hours ago
Last Refreshed
Books and Lecture NotesPapersImplementation in JAX
Use this list with your AI agent
Add the Context Awesome MCP server to Claude, Cursor, or any MCP client, then ask:
"Show me diagonal scaling resources from awesome-soms"
Installation instructions →What's inside
Papers
- AdaHessian: An Adaptive Second Order Optimizer for Machine LearningDiagonal Scaling
- A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to RegularizationAnalysis of the Hessian
- A Multi-Batch L-BFGS Method for Machine LearningQuasi-Newton
- A Stochastic Quasi-Newton Method for Large-Scale OptimizationQuasi-Newton
- Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural NetworksGauss-Newton
- Empirical Analysis of the Hessian of Over-Parametrized Neural NetworksAnalysis of the Hessian
Implementation in JAX
- AdaHessianJax
implementation of the AdaHessian optimizer by Nestor Demeure
- JAXopt
deterministic second-order methods (e.g., Gauss-Newton, Levenberg Marquardt), stochastic first-order methods PolyakSGD, ArmijoSGD
- KFAC-JAX
implementation of KFAC from the DeepMind team
- Optax
mostly first-order accelerated methods
- Somax
second-order stochastic solvers
Showing a sample of 32 resources. View the full list on GitHub →