Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Scanpath and saliency prediction on 360 degree images

Assens, Marc, Giró-i-Nieto, Xavier orcid logoORCID: 0000-0002-9935-5332, McGuinness, Kevin orcid logoORCID: 0000-0003-1336-6477 and O'Connor, Noel E. orcid logoORCID: 0000-0002-4033-9135 (2018) Scanpath and saliency prediction on 360 degree images. Signal Processing: Image Communication, 69 . pp. 8-14. ISSN 0923-5965

Abstract
We introduce deep neural networks for scanpath and saliency prediction trained on 360-degree images. The scanpath prediction model called SaltiNet is based on a temporal-aware novel representation of saliency information named the saliency volume. The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation using a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes. Sampling strategies over these volumes are used to generate scanpaths over the 360-degree images. Our experiments show the advantages of using saliency volumes, and how they can be used for related tasks. We also show how a similar architecture achieves state-of-the-art performance for the related task of saliency map prediction. Our source code and trained models available at https://github.com/massens/saliency-360salient-2017.
Metadata
Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:Deep learning; saliency; scanpath; visual attention
Subjects:Computer Science > Artificial intelligence
Computer Science > Image processing
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Publisher:Elsevier
Official URL:https://doi.org/10.1016/j.image.2018.06.006
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:TEC2013- 290 43935-R and TEC2016-75976-R, funded by the Spanish Ministerio de Economia y Competitividad and the European Regional Development Fund (ERDF), SGR14 Consolidated Research Group recognized and sponsored by AGAUR, the Catalan Government (Generalitat de Catalunya), Science Foundation Ireland under Grant No 15/SIRG/3283.
ID Code:22817
Deposited On:03 Dec 2018 14:29 by Kevin Mcguinness . Last Modified 31 May 2019 08:22
Documents

Full text available as:

[thumbnail of Accepted Manuscript]
Preview
PDF (Accepted Manuscript) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
2MB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record