Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Pseudo-labeling and confirmation bias in deep semi-supervised learning

Arazo Sánchez, Eric, Ortego, Diego orcid logoORCID: 0000-0002-1011-3610, Albert, Paul, O'Connor, Noel E. orcid logoORCID: 0000-0002-4033-9135 and McGuinness, Kevin orcid logoORCID: 0000-0003-1336-6477 (2020) Pseudo-labeling and confirmation bias in deep semi-supervised learning. In: 2020 International Joint Conference on Neural Networks, 19 - 24 July 2020, Glasgow, Scotland. ISBN 978-1-7281-6926-2

Abstract
Semi-supervised learning, i.e. jointly learning from labeled and unlabeled samples, is an active research topic due to its key role on relaxing human supervision. In the context of image classification, recent advances to learn from unlabeled samples are mainly focused on consistency regularization methods that encourage invariant predictions for different perturbations of unlabeled samples. We, conversely, propose to learn from unlabeled data by generating soft pseudo-labels using the network predictions. We show that a naive pseudo-labeling overfits to incorrect pseudo-labels due to the so-called confirmation bias and demonstrate that mixup augmentation and setting a minimum number of labeled samples per mini-batch are effective regularization techniques for reducing it. The proposed approach achieves state-of-the-art results in CIFAR-10/100, SVHN, and Mini-ImageNet despite being much simpler than other methods. These results demonstrate that pseudo-labeling alone can outperform consistency regularization methods, while the opposite was supposed in previous work. https://git.io/fjQsC
Metadata
Item Type:Conference or Workshop Item (Paper)
Event Type:Conference
Refereed:Yes
Subjects:Computer Science > Artificial intelligence
Computer Science > Image processing
Computer Science > Machine learning
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Electronic Engineering
Research Initiatives and Centres > INSIGHT Centre for Data Analytics
Published in: 2020 International Joint Conference on Neural Networks (IJCNN). . IEEE. ISBN 978-1-7281-6926-2
Publisher:IEEE
Official URL:http://dx.doi.org/10.1109/IJCNN48605.2020.9207304
Copyright Information:© 2020 The Authors
Use License:This item is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 3.0 License. View License
Funders:Science Foundation Ireland (SFI) under grant number SFI/15/SIRG/3283, Science Foundation Ireland (SFI) under grant number SFI/12/RC/2289 P2., Insight
ID Code:24371
Deposited On:22 Jul 2020 12:38 by Kevin Mcguinness . Last Modified 28 Apr 2022 10:25
Documents

Full text available as:

[thumbnail of PID6429195.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
811kB
Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record