30–31 May 2024
Wigner Datacenter - HUN-REN Wigner Research Centre for Physics
Europe/Budapest timezone

Generalization bounds via estimation of tangent sensitivity tensors of ReLU networks and of the H2 norm of neural ODEs and SSMs

31 May 2024, 16:10
25m
Wigner Datacenter - HUN-REN Wigner Research Centre for Physics

Wigner Datacenter - HUN-REN Wigner Research Centre for Physics

HUN-REN Wigner RCP 1121 Budapest, Konkoly-Thege Miklós rd 29-33, Hungary

Speaker

Balint Daroczy (HUN-REN SZTAKI)

Description

We consider generalization bounds for two types of neural structures, feedforward Rectified Linear Unit (ReLU) networks, special types of neural Ordinary Differential Equations (ODE) and State Space Models (SSM). Calculating the Rademacher complexity of both models involves computationally expensive norm calculations therefore we propose techniques to compute them efficiently.

Primary author

Balint Daroczy (HUN-REN SZTAKI)

Presentation materials