Séminaire de Probabilités et Statistique :

Le 20 juin 2022 à 13:45 - UM - Bât 09 - Salle de conférence (1er étage)

Présentée par Dieuleveut Aymeric - CMAP, École Polytechnique

Federated Learning and optimization: from a gentle introduction to recent results

In this presentation, I will present some results on optimization in the context of federated learning. I will first summarise the main challenges and the type of results the community has obtained, and dive into some more recent results on tradeoffs between (bidirectional) compression, communication, privacy and user-heterogeneity. In particular, I will describe two fundamental phenomenons (and related proof techniques): (1) how user-heterogeneity affects convergence of federated optimization methods in the presence of communication constraints, and (2) the robustness of distributed stochastic algorithms to perturbation of the iterates, and the link with model compression.

The presentation will be based on recent work with Constantin Philippenko, Gersende Fort, Eric Moulines, Geneviève Robin.

Main references:
- Preserved central model for faster bidirectional compression in distributed settings C. Philippenko, A. Dieuleveut, Neurips 2021
https://proceedings.neurips.cc/paper/2021/file/13d63838ef1fb6f34ca2dc6821c60e49-Paper.pdf
- Federated Expectation Maximization with heterogeneity mitigation and variance reduction, A. Dieuleveut, G. Fort, E. Moulines, G. Robin, Neurips 2021
https://proceedings.neurips.cc/paper/2021/file/f740c8d9c193f16d8a07d3a8a751d13f-Paper.pdf

Séminaire à 13h45 en salle 109 (IMAG, bâtiment 9).
Également retransmis sur zoom : https://umontpellier-fr.zoom.us/j/94087408185