Séminaire de Probabilités et Statistique
lundi 20 juin 2022 à 13:45 - UM - Bât 09 - Salle de conférence (1er étage)
Aymeric Dieuleveut (CMAP, École Polytechnique)
Federated Learning and optimization: from a gentle introduction to recent results
In this presentation, I will present some results on optimization in the context of federated learning. I will first summarise the main challenges and the type of results the community has obtained, and dive into some more recent results on tradeoffs between (bidirectional) compression, communication, privacy and user-heterogeneity. In particular, I will describe two fundamental phenomenons (and related proof techniques): (1) how user-heterogeneity affects convergence of federated optimization methods in the presence of communication constraints, and (2) the robustness of distributed stochastic algorithms to perturbation of the iterates, and the link with model compression.
The presentation will be based on recent work with Constantin Philippenko, Gersende Fort, Eric Moulines, Geneviève Robin.
Main references:
- Preserved central model for faster bidirectional compression in distributed settings C. Philippenko, A. Dieuleveut, Neurips 2021
https://proceedings.neurips.cc/paper/2021/file/13d63838ef1fb6f34ca2dc6821c60e49-Paper.pdf
- Federated Expectation Maximization with heterogeneity mitigation and variance reduction, A. Dieuleveut, G. Fort, E. Moulines, G. Robin, Neurips 2021
https://proceedings.neurips.cc/paper/2021/file/f740c8d9c193f16d8a07d3a8a751d13f-Paper.pdf
Séminaire à 13h45 en salle 109 (IMAG, bâtiment 9).
Également retransmis sur zoom :
https://umontpellier-fr.zoom.us/j/94087408185