Smeaton, Alan F. ORCID: 0000-0003-1028-8389 (2020) Generative forms of multimedia content. (Opening keynote talk). In: IEEE International Conference on Multimedia and Expo - ICME 2020, 6-9 July 2020, London, UK (virtual).
Abstract
Multimedia analysis has recently made spectacular improvements in both quality and in sophistication. Over the last half-decade we have seen extreme progress in tasks like image and video tagging, object detection and activity recognition, generating descriptive captions and more. Some of these have been deployed and are in widespread use in our smartphones and on our social media platforms. We have also seen recent research work, including our own, on computing more abstract features of multimedia, such as person-counting from CCTV, computing visual salience, estimating aesthetics of images and videos, and computing video memorability.
The common methodology used across most of these applications is of course machine learning, in all its forms, from convolutional neural networks to simple regression and support vector machines. Much of the research in our field is about wrestling with machine learning to optimise its performance in multimedia analysis tasks and this recent run of extreme progress does not look like ending anytime soon, though it will reach its high water mark. When it does reach the point at which it cannot get any better, what then ?
Generative machine learning (ML) is a recent form of media analysis which turns the conventional approach on its head and its methodology is to train a model and then generate new data. Example applications of generative ML deoldify which colourises black and white images and video clips, and Generative Adversarial Networks (GANs) which can generate DNA sequences, 3D models of replacement teeth, impressionist paintings, and of course video clips, some known as deepfakes. Putting aside the more nefarious applications of deepfakes, what is the potential for generative forms of multimedia ? In the short to medium term we can speculate that it would include things like movie augmentation but it how far can it go and could it replicate human creativity ?
In this talk I will introduce some of the recent forms of generative multimedia and discuss how far I believe we could go with this exciting new technology.
Metadata
Item Type: | Conference or Workshop Item (Invited Talk) |
---|---|
Event Type: | Conference |
Refereed: | No |
Additional Information: | Video recording of the presentation is available at https://2020.ieeeicme-virtual.org/presentation/keynote/keynote-1-generative-forms-multimedia-content-alan-smeaton |
Subjects: | Computer Science > Multimedia systems |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Initiatives and Centres > INSIGHT Centre for Data Analytics |
Copyright Information: | © 2020 The Author |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License |
Funders: | Science Foundation Ireland |
ID Code: | 24751 |
Deposited On: | 08 Jul 2020 16:21 by Alan Smeaton . Last Modified 18 Dec 2020 17:13 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
104MB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record