ERA: Enhanced Rational Activations

Martin Trimmel, Mihai Zanfir, Richard Hartley, Cristian Sminchisescu

Forskningsoutput: Kapitel i bok/rapport/Conference proceedingKonferenspaper i proceedingPeer review

Sammanfattning

Activation functions play a central role in deep learning since they form an essential building stone of neural networks. In the last few years, the focus has been shifting towards investigating new types of activations that outperform the classical Rectified Linear Unit (ReLU) in modern neural architectures. Most recently, rational activation functions (RAFs) have awakened interest because they were shown to perform on par with state-of-the-art activations on image classification. Despite their apparent potential, prior formulations are either not safe, not smooth, or not “true” rational functions, and they only work with careful initialisation. Aiming to mitigate these issues, we propose a novel, enhanced rational function, ERA, and investigate how to better accommodate the specific needs of these activations, to both network components and training regime. In addition to being more stable, the proposed function outperforms other standard ones across a range of lightweight network architectures on two different tasks: image classification and 3d human pose and shape reconstruction.

Originalspråkengelska
Titel på värdpublikationComputer Vision – ECCV 2022 - 17th European Conference, Proceedings
RedaktörerShai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner
FörlagSpringer Science and Business Media B.V.
Sidor722-738
Antal sidor17
ISBN (tryckt)9783031200434
DOI
StatusPublished - 2022
Evenemang17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel
Varaktighet: 2022 okt. 232022 okt. 27

Publikationsserier

NamnLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volym13680 LNCS
ISSN (tryckt)0302-9743
ISSN (elektroniskt)1611-3349

Konferens

Konferens17th European Conference on Computer Vision, ECCV 2022
Land/TerritoriumIsrael
OrtTel Aviv
Period2022/10/232022/10/27

Ämnesklassifikation (UKÄ)

  • Datavetenskap (datalogi)

Fingeravtryck

Utforska forskningsämnen för ”ERA: Enhanced Rational Activations”. Tillsammans bildar de ett unikt fingeravtryck.

Citera det här