Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

TinyHD: Efficient video saliency prediction with heterogeneous decoders using hierarchical maps distillation

Hu, Feiyan orcid logoORCID: 0000-0001-7451-6438, Palazzo, Simone orcid logoORCID: 0000-0002-2441-0982, Proietto Salanitri, Federica orcid logoORCID: 0000-0002-6122-4249, Bellitto, Giovanni, Moradi, Morteza, Spampinato, Concetto orcid logoORCID: 0000-0001-6653-2577 and McGuinness, Kevin orcid logoORCID: 0000-0003-1336-6477 (2023) TinyHD: Efficient video saliency prediction with heterogeneous decoders using hierarchical maps distillation. In: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023, 3-7 Jan 2023, Waikoloa, Hawaii.

Abstract
Video saliency prediction has recently attracted atten- tion of the research community, as it is an upstream task for several practical applications. However, current so- lutions are particurly computationally demanding, espe- cially due to the wide usage of spatio-temporal 3D convolu- tions. We observe that, while different model architectures achieve similar performance on benchmarks, visual varia- tions between predicted saliency maps are still significant. Inspired by this intuition, we propose a lightweight model that employs multiple simple heterogeneous decoders and adopts several practical approaches to improve accuracy while keeping computational costs low, such as hierarchi- cal multi-map knowledge distillation, multi-output saliency prediction, unlabeled auxiliary datasets and channel re- duction with teacher assistant supervision. Our approach achieves saliency prediction accuracy on par or better than state-of-the-art methods on DFH1K, UCF-Sports and Hol- lywood2 benchmarks, while enhancing significantly the ef- ficiency of the model.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Subjects:Computer Science > Artificial intelligence
Computer Science > Machine learning
Computer Science > Digital video
Engineering > Signal processing
Engineering > Electronic engineering
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Published in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). . IEEE.
Publisher:IEEE
Official URL:https://doi.org/10.1109/WACV56688.2023.00209
Copyright Information:© 2023 IEEE
Funders:Science Foundation Ireland (SFI) under grant number SFI/12/RC/2289 P2, Regione Sicilia, Italy, RehaStart project (grant identifier: PO FESR 2014/2020, Azione 1.1.5, N. 08ME6201000222, CUP G79J18000610007), University of Catania, Piano della Ricerca di Ateneo, 2020/2022,Linea2D, MIUR,Italy,Azione1.2“Mobilita` dei Ricercatori” (grant identifier: Asse I, PON R&I 2014- 2020, id. AIM 1889410, CUP: E64I18002520007)
ID Code:27962
Deposited On:09 Jan 2023 14:10 by Feiyan Hu . Last Modified 16 Nov 2023 13:46
Documents

Full text available as:

[thumbnail of 0425.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Creative Commons: Attribution-Noncommercial-Share Alike 4.0
3MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record