סמינר מחלקת מערכות - Dr. Ofir Lindenbaum (BIU) - Exploiting Randomness in Machine Learning
(The talk will be given in English)
Speaker: Prof. Ofir Lindenbaum
Faculty of Engineering, Bar Ilan University
011 hall, Electrical Engineering-Kitot Building
Monday, May 15th, 2023
15:00 - 16:00
Exploiting Randomness in Machine Learning
Abstract
Noise plays a central role in many machine learning algorithms; one well-studied example is stochastic gradient descent (SGD). Despite its simplicity, i.e., being a noisy first-order optimization method, SGD empirically outperforms gradient descent (GD) and second-order methods. In the first part of the talk, I will present Gaussian Stochastic Gates (STGs), a differentiable non-convex relaxation of the L0 norm highly effective for feature selection. Under a linear sparse regression model, I will demonstrate that our STG probabilistic feature selection formulation can recover informative features successfully. Then, I will show how a randomly aggregated least squares procedure can improve the probability of exact recovery. In the second part of the talk, I will revisit the noise model of SGD and present strong evidence that it is well characterized by a symmetric $S\alpha S$ Lévy distribution. Then, by modeling the training process as Lévy-driven stochastic differential equations (SDEs), where each parameter distributes with a different $\alpha$ value, we analyze different properties of deep nets near local minima.
(based and join works with Yutaro Tamada, Yuval Kluger, Sahand Negahban, and Barak Battash)
Short Bio
Ofir is a senior lecturer in the faculty of Engineering at Bar Ilan University. Ofir obtained his Ph.D. and M.Sc. from Tel Aviv University and his B.Sc. in Electrical Engineering and Physics (both summa cum laude) from the Technion. Following his Ph.D., he served as a Gibbs assistant professor at Yale University. His research is focused on the theory and practice of machine learning. His main goal is to enable the practical use of machine learning algorithms for scientific discovery.
השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום שם מלא + מספר ת.ז. בטופס הנוכחות שיועבר באולם במהלך הסמינר
