Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

The influence of audio on video memorability with an audio gestalt regulated video memorability system

Sweeney, Lorin orcid logoORCID: 0000-0002-3427-1250, Healy, Graham orcid logoORCID: 0000-0001-6429-6339 and Smeaton, Alan F. orcid logoORCID: 0000-0003-1028-8389 (2021) The influence of audio on video memorability with an audio gestalt regulated video memorability system. In: 18th conference of Content-Based Multimedia Indexing, 28-30 June 2021, Lille, France (Online).

Abstract
Memories are the tethering threads that tie us to the world, and memorability is the measure of their tensile strength. The threads of memory are spun from fibres of many modalities, obscuring the contribution of a single fibre to a thread's overall tensile strength. Unfurling these fibres is the key to understanding the nature of their interaction, and how we can ultimately create more meaningful media content. In this paper, we examine the influence of audio on video recognition memorability, finding evidence to suggest that it can facilitate overall video recognition memorability rich in high-level (gestalt) audio features. We introduce a novel multimodal deep learning-based late-fusion system that uses audio gestalt to estimate the influence of a given video's audio on its overall short-term recognition memorability, and selectively leverages audio features to make a prediction accordingly. We benchmark our audio gestalt based system on the Memento10k short-term video memorability dataset, achieving top-2 state-of-the-art results.
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Uncontrolled Keywords:Memorability; multimodal; audio gestalt; deep learning
Subjects:Computer Science > Machine learning
Computer Science > Multimedia systems
Computer Science > Digital video
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Published in: 2021 International Conference on Content-Based Multimedia Indexing (CBMI). .
Official URL:https://dx.doi.org/10.1109/CBMI50038.2021.9461903
Copyright Information:© 2021 IEEE
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Science Foundation Ireland (SFI) Grant Number SFI/12/RC/2289 P2, European Regional Development Fund
ID Code:25801
Deposited On:28 Jun 2021 13:11 by Lorin Sweeney . Last Modified 05 Jul 2021 09:59
Documents

Full text available as:

[thumbnail of CBMI_2021_Memorability_Lorin.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
632kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record