Séminaire de Probabilités et Statistique :

Le 15 mai 2023 à 14:00 - UM - Bât 09 - Salle de conférence (1er étage)

Présentée par Portier François - CREST, ENSAI

SGD with Coordinate Sampling: Theory and Practice

While classical forms of stochastic gradient descent algorithm treat the different coordinates in the same way, a framework allowing for adaptive (non uniform) coordinate sampling is developed to leverage structure in data. In a non-convex setting and including zeroth order gradient estimate, almost sure convergence as well as non-asymptotic bounds are established. Within the proposed framework, we develop an algorithm, MUSKETEER, based on a reinforcement strategy: after collecting information on the noisy gradients, it samples the most promising coordinate (all for one); then it moves along the one direction yielding an important decrease of the objective (one for all). Numerical experiments on both synthetic and real data examples confirm the effectiveness of MUSKETEER in large scale problems.

Joint work with Rémi Leluc, Postdoc at école Polytechnique. https://arxiv.org/abs/2105.11818

Attention: séminaire à 14h
En salle 109, également retransmis sur zoom : https://umontpellier-fr.zoom.us/j/94087408185