BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//132.216.98.100//NONSGML kigkonsult.se iCalcreator 2.20.4//
BEGIN:VEVENT
UID:20250811T052224EDT-5981eGixcg@132.216.98.100
DTSTAMP:20250811T092224Z
DESCRIPTION:Title: Normalization effects on deep neural networks and deep l
 earning for scientific problems.\n\nAbstract:\n\nWe study the effect of no
 rmalization on the layers of deep neural networks. A given layer ii with N
 iNi hidden units is normalized by 1/Nγii1/Niγi with γi∈[1/2\,1]γi∈[1/2\,1]
 . We study the effect of the choice of the γiγi on the statistical behavio
 r of the neural network’s output (such as variance) as well as on the test
  accuracy and generalization properties of the architecture. We find that 
 in terms of variance of the neural network’s output and test accuracy the 
 best choice is to choose the γiγi’s to be equal to one\, which is the mean
 -field scaling. We also find that this is particularly true for the outer 
 layer\, in that the neural network’s behavior is more sensitive in the sca
 ling of the outer layer as opposed to the scaling of the inner layers. The
  mechanism for the mathematical analysis is an asymptotic expansion for th
 e neural network’s output. An important practical consequence of the analy
 sis is that it provides a systematic and mathematically informed way to ch
 oose the learning rate hyperparameters. Such a choice guarantees that the 
 neural network behaves in a statistically robust way as the number of hidd
 en units NiNi grow. Time permitting\, I will discuss applications of these
  ideas to design of deep learning algorithms for scientific problems inclu
 ding solving high dimensional partial differential equations (PDEs)\, clos
 ure of PDE models and reinforcement learning with applications to financia
 l engineering\, turbulence and more.\n\nSpeaker\n\nKonstantinos Spiliopoul
 os is the Director of Statistics and a Professor of Mathematics and Statis
 tics at Boston University (BU). He is a member of the Hariri Institute for
  Computing and of the Center for Information and Systems Engineering at BU
 . Between 2009-2012\, he was a Prager Assistant Professor at the Division 
 of Applied Mathematics at Brown University\, and he earned his PhD in Math
 ematical Statistics at University of Maryland in 2009. He has been at BU s
 ince 2012. Currently\, Professor Spiliopoulos works on mathematical and co
 mputational methods for machine learning algorithms\, stochastic online al
 gorithms and deep learning for scientific problems\, accelerated Monte Car
 lo methods for sampling and optimization\, agent-based modeling and applic
 ations to financial engineering\, biological systems and opinion and socia
 l dynamics. He has received the Stochastics and Dynamics Best Paper Award 
 in 2023\, the Simons Fellow in Mathematics Award in 2020\, the NSF Career 
 Award in 2016\, the Hariri Institute Junior Fellowship Award in 2013\, the
  Seymour Goldberg Paper Award in 2008 and the Eygenidio Foundation Award i
 n 2004. His work has been continuously funded by NSF\, Simons Foundations 
 and the DoD. He has co-authored the book “Mathematical Foundations of Deep
  Learning” to appear later in 2025.\n\nhttps://mcgill.zoom.us/j/8110065421
 2\n\nMeeting ID: 811 0065 4212\n\nPasscode: None\n\n \n
DTSTART:20250404T193000Z
DTEND:20250404T203000Z
LOCATION:Room 1104\, Burnside Hall\, CA\, QC\, Montreal\, H3A 0B9\, 805 rue
  Sherbrooke Ouest
SUMMARY:Konstantinos Spiliopoulos (Boston University)
URL:/mathstat/channels/event/konstantinos-spiliopoulos
 -boston-university
END:VEVENT
END:VCALENDAR