Oliveira, Marlon (2018) Handshape recognition using principal component analysis and convolutional neural networks applied to sign language. PhD thesis, Dublin City University.
Abstract
Handshape recognition is an important problem in computer vision with significant societal impact. However, it is not an easy task, since hands are naturally deformable objects. Handshape recognition contains open problems, such as low accuracy or low speed, and despite a large number of proposed approaches, no solution has been found to solve these open problems. In this thesis, a new image dataset for Irish Sign Language (ISL) recognition is introduced. A deeper study using only 2D images is presented on Principal Component Analysis (PCA) in two stages. A comparison between approaches that do not need features (known as end-to-end) and feature-based approaches is carried out.
The dataset was collected by filming six human subjects performing ISL handshapes and movements. Frames from the videos were extracted. Afterwards the redundant images were filtered with an iterative image selection process that selects the images which keep the dataset
diverse.
The accuracy of PCA can be improved using blurred images and interpolation. Interpolation is only feasible with a small number of points. For this reason two-stage PCA is proposed. In other words, PCA is applied to another PCA space. This makes the interpolation possible and improves the accuracy in recognising a shape at a translation and rotation unknown in the training stage.
Finally classification is done with two different approaches: (1) End-to-end approaches and (2) feature-based approaches. For (1) Convolutional Neural Networks (CNNs) and other classifiers are tested directly over raw pixels, whereas for (2) PCA is mostly used to extract features and again different algorithms are tested for classification. Finally, results are presented showing accuracy and speed for (1) and (2) and how blurring affects the accuracy.
Metadata
Item Type: | Thesis (PhD) |
---|---|
Date of Award: | January 2018 |
Refereed: | No |
Supervisor(s): | Sutherland, Alistair |
Uncontrolled Keywords: | Computer Vision; Deep Learning; Principal Component Analysis; Image Recognition; Sign Language |
Subjects: | Computer Science > Machine learning Computer Science > Image processing |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing |
Use License: | This item is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 3.0 License. View License |
Funders: | Science Without Borders CAPES- Brazilian Program - Process: 9064-13-3 |
ID Code: | 22191 |
Deposited On: | 05 Apr 2018 09:34 by Alistair Sutherland . Last Modified 19 Jul 2018 15:12 |
Documents
Full text available as:
Preview |
PDF (PhD Thesis)
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
10MB |
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record