Nov 17 – 18, 2022
Mercure Budapest Castle Hill
Europe/Budapest timezone

On the Role of Neural Collapse in Transfer and Few-Shot Learning

Not scheduled
20m
Mercure Budapest Castle Hill

Mercure Budapest Castle Hill

Budapest, Ntak:Sz19000364, Szálloda, Krisztina krt. 41-43, 1013•(06 1) 488 8100

Speaker

András György (Deepmind)

Description

Using representations learned by large, pretrained models, also called foundation models, in new tasks with fewer data has been successful in a wide range of machine learning problems. In particular, recent results in the literature show that representations learned by a single classifier over many classes are competitive on few-shot learning problems with representations learned by special-purpose algorithms designed for such problems. In this talk, I will provide a theoretical explanation for this behavior based on the recently observed phenomenon that the features learned by overparameterized classification networks show an interesting clustering property, called neural collapse.

Based on joint work with Tomer Galanti and Marcus Hutter.

Primary author

Presentation materials

There are no materials yet.