Afli, Haithem ORCID: 0000-0002-7449-4707, Hu, Feiyan ORCID: 0000-0001-7451-6438, Du, Jinhua ORCID: 0000-0002-3267-4881, Cosgrove, Daniel, McGuinness, Kevin ORCID: 0000-0003-1336-6477, O'Connor, Noel E. ORCID: 0000-0002-4033-9135, Arazo Sánchez, Eric, Zhou, Jiang ORCID: 0000-0002-3067-8512 and Smeaton, Alan F. ORCID: 0000-0003-1028-8389 (2017) Dublin City University participation in the VTT track at TRECVid 2017. In: TRECVid workshop, 13-15 Nov 2017, Gaithersburg, Md., USA.
Abstract
Dublin City University participated in the video-to-text caption generation task in TRECVid and this paper describes the three approaches we took for our 4 submitted runs. The first approach is based on extracting regularly-spaced keyframes from a video, generating a text caption for each keyframe and then combining the keyframe captions into a single caption. The second approach is based on detecting image crops from those keyframes using saliency map to include as much of the attractive part of the image as possible, generating a caption for each crop in each keyframe, and combining the captions into one. The third approach is an end-to-end system, a true deep learning submission based on MS-COCO, an externally available set of training captions. The paper presents a description and the official results of each of the approaches.
Metadata
Item Type: | Conference or Workshop Item (Poster) |
---|---|
Event Type: | Workshop |
Refereed: | No |
Uncontrolled Keywords: | Video captions |
Subjects: | Computer Science > Computational linguistics Computer Science > Artificial intelligence Computer Science > Digital video Computer Science > Image processing |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering Research Initiatives and Centres > INSIGHT Centre for Data Analytics Research Initiatives and Centres > ADAPT |
Copyright Information: | © 2017 The Authors |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | Science Foundation Ireland, SFI/12/RC/2289 (Insight Centre), Science Foundation Ireland, SFI/13/RC/2106 (ADAPT Centre). |
ID Code: | 22155 |
Deposited On: | 04 Jan 2018 10:05 by Alan Smeaton . Last Modified 28 Apr 2022 10:27 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
2MB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record