Schedule, Notes/Handouts & Videos

Optimal transport (OT) has recently gained lot of interest in machine learning. It is a natural tool to compare in a geometrically faithful way probability distributions. It finds applications in both supervised learning (using geometric loss functions) and unsupervised learning (to perform generative model fitting). OT is however plagued by the curse of dimensionality, since it might require a number of samples which grows exponentially with the dimension. In this talk, I will review entropic regularization methods which define geometric loss functions approximating OT with a better sample complexity. More information and references can be found on the website of our book "Computational Optimal Transport" https://optimaltransport.github.io/

Optimal transport is a powerful tool for measuring the distances between signals. However, the most common choice is to use the Wasserstein distance where one is required to treat the signal as a probability measure. This places restrictive conditions on the signals and although ad-hoc renormalisation can be applied to sets of unnormalised measures this can often dampen features of the signal. The second disadvantage is that despite recent advances, computing optimal transport distances for large sets is still difficult. In this talk I will focus on the Hellinger-Kantorovich distance, which can be applied between any pair of non-negative measures. I will describe how the distance can be linearised and embedded into a Euclidean space (the analogue of the linear optimal transport framework for Hellinger-Kantorovich). The Euclidean distance in the embedded space is approximately the Wasserstein distance in the original space. This method, in particular, allows for the application of off-the-shelf data analysis tools such as principal component analysis.

This is joint work with Bernhard Schmitzer (TU Munich).

In this talk, I will present rates of convergence for Wasserstein barycenters using gradient descent and stochastic gradient descent. While the barycenter functional is not geodesically convex, this result hinges on a Polyak-Lojasiewicz (PL) inequality in the case where the underlying distribution is supported on a subset of Gaussian distributions.

In this talk I will discuss some recent results on the asymptotic behaviour of a family of weighted ultrafast diffusion PDEs. These equations are motivated by the gradient flow approach to the problem of quantization of measures, introduced in a series of joint papers with Emanuele Caglioti and François Golse. In this presentation I will focus on a result obtained in collaboration with Francesco Saverio Patacchini and Filippo Santambrogio, where we use the JKO scheme to obtain existence, uniqueness, and exponential convergence to equilibrium under minimal assumptions on the data.

We consider two sharp next-order asymptotics problems, namely the asymptotics for the minimum energy for optimal point configurations and the asymptotics for the many-marginals Optimal Transport, in both cases with Riesz costs with inverse power-law long range interactions. The first problem describes the ground state of a Coulomb or Riesz gas, while the second appears as a semiclassical limit of the Density Functional Theory energy modelling a quantum version of the same system. Recently the second-order term in these expansions was precisely described, and corresponds respectively to a Jellium and to a Uniform Electron Gas model. The present work shows that for inverse-power-law interactions with power s∈[d−2,d) in d dimensions, the two problems have the same minimum value asymptotically. For the Coulomb case in d=3, our result verifies the physicists' long-standing conjecture regarding the equality of the second-order terms for these two problems. Furthermore, our work implies that, whereas minimum values are equal, the minimizers may be different. Moreover, provided that the crystallization hypothesis in d=3 holds, which is analogous to Abrikosov's conjecture in d=2, then our result verifies the physicists' conjectured ≈1.4442 lower bound on the famous Lieb-Oxford constant. Our work also rigorously confirms some of the predictions formulated by physicists, regarding the optimal value of the Uniform Electron Gas second-order asymptotic term. We also show that on the whole range s∈(0,d), the Uniform Electron Gas second-order constant is continuous in s.

Entropy-regularized OT (EOT) was first introduced by Cuturi in 2013 as a solution to the computational burden of OT for machine learning problems. In this talk, after studying the properties of EOT, we will introduce a new family of losses between probability measures called Sinkhorn Divergences. Based on EOT, this family of losses actually interpolates between OT (no regularization) and MMD (infinite regularization). We will illustrate these theoretical claims on a set of learning problems formulated as minimizations over the space of measures.

In this talk, we propose an analysis of gradient descent on wide two-layer ReLU neural networks that leads to sharp characterizations of the learned predictor and strong generalization performances. The main idea is to study the dynamics when the width of the hidden layer goes to infinity, which is a Wasserstein gradient flow. While this dynamics evolves on a non-convex landscape, we show that its limit is a global minimizer if initialized properly. We also study the "implicit bias" of this algorithm when the objective is the unregularized logistic loss. We finally discuss what these results tell us on the generalization performance. This is based on joint work with Francis Bach.

Modern neural networks contain millions of parameters, and training them requires to optimize a highly non-convex objective. Despite the apparent complexity of this task, practitioners successfully train such models using simple first order methods such as stochastic gradient descent (SGD). I will survey recent efforts to understand this surprising phenomenon using tools from the theory of partial differential equations. Namely, I will discuss a mean field limit in which the number of neurons becomes large, and the SGD dynamics is approximated by a certain Wasserstein gradient flow. [Joint work with Adel Javanmard, Song Mei, Theodor Misiakiewicz, Marco Mondelli, Phan-Minh Nguyen]

We consider transport equations on graphs, where mass is distributed over vertices and is transported along the edges. The first part of the talk will deal with the graph analogue of the Wasserstein distance, in the particular case where the notion of density along edges is inspired by the upwind numerical schemes. While this approach has both theoretical and computational advantages, the resulting distance is only a quasi-metric. We investigate this quasi-metric both on graphs and on more general structures where the set of ``vertices'' is an arbitrary positive measure. In the second part of the talk we will interpret the nonlocal-interaction equation equations on graphs as gradient flows with respect to the graph-Wasserstain quasi-metric of the nonlocal-interaction energy. We show that for graphs representing data sampled from a manifold, the solutions of the nonlocal-interaction equations on graphs converge to solutions of an integral equation on the manifold.