April 14, 2020

12:00 pm / 1:00 pm


Seminar will be remote via Zoom

Tuesday, April 14, 2020 at12:00 pm EST

Seminar will be remote viaZoom
Meeting ID: 255 489 222
Password: clark_hall

?Learning stepsizes for unfolded sparse coding?

Thomas Moreau,PhD

INRIA Saclay


Abstract:Sparse coding is typically solved by iterative optimization techniques,such as the Iterative Shrinkage-Thresholding Algorithm (ISTA). Unfolding andlearning weights of ISTA using neural networks is apractical way to accelerateestimation. However, the reason why learning the weights of such a networkwould accelerate sparse coding are not clear. In this talk, we look at thisproblem from the point of view of selecting adapted step sizes for ISTA. Weshow that a simple step size strategy can improve the convergence rate of ISTAby leveraging the sparsity of the iterates.However, it is impractical in mostlarge-scale applications. Therefore, we propose a network architecture whereonly the step sizes of ISTA are learned. We demonstrate if the learned algorithmconverges to the solution of theLasso, its last layers correspond to ISTA withlearned step sizes. Experiments show that learning step sizes can effectivelyaccelerate the convergence when the solutions are sparse enough.

Bio:Thomas Moreau received the graduate degree from the Ecole Polytechnique,Palaiseau, France,in 2014, and the PhD degree from the Ecole NormaleSup√©rieure, Cachan, France, in 2017 under the supervision of Nicolas Vayatisand Laurent Oudre in the CMLA laboratory. He recently joined the Inria Parietalproject team inSaclay, first as a post-doctoral researcher and then as aresearcher. His research interests include unsupervised learning, image/signalprocessing and distributed computing.