Інерційна система розпізнавання жестів

Maksym Oleksandrovych Yaroshenko; Anton Yuriiovych Varfolomieiev; Petro Oleksiiovych Yaganov

doi:10.20535/2523-4455.2019.24.5.193295

Vol. 24 · Issue 5 · 2019 Oct 31, 2019 Electronic Systems and Signals

The Inertial Gesture Recognition System

Maksym Oleksandrovych Yaroshenko Corresponding Національний технічний університет України "Київський політехнічний інститут імені Ігоря Сікорського" Ukraine

Anton Yuriiovych Varfolomieiev Національний технічний університет України "Київський політехнічний інститут імені Ігоря Сікорського" Ukraine

Petro Oleksiiovych Yaganov Національний технічний університет України "Київський політехнічний інститут імені Ігоря Сікорського" Ukraine

Pages42-47 PublishedOct 31, 2019

PDF (Ukrainian)

VOL 24 · ISS 5 · 2019 View issue

Abstract

The paper is devoted to the implementation of the gestures recognition system. Existing gesture recognition approaches are considered. Among them the system of inertial type is chosen for research, since the systems of this type are less difficult to implement and require relatively low computational complexity. Thus, the proposed system is based on two components: the hardware module for gesture capturing and the subsystem for the recognition of captured gestures. The hardware component of the system employs MEMS accelerometer and gyroscope for motion acquisition. The prototype of this component is implemented using the STM32F401RE microcontroller and the MEMS sensor MPU9250. The communication between the microcontroller and the MEMS sensor is carried out via the I2C bus. The microcontroller initializes the sensor and acquires the motion data. Then it sends the collected data via the virtual COM port to the PC, where the captured motions are recognized. The system component for the recognition of gestures is implemented as the neural network in Matlab environment. This network consists of four layers: the input layer, the BLSTM layer, the fully-connected layer, and the Softmax layer. The key component of the selected neural network architecture is the BLSTM layer, which (due to its properties) provides some invariance to the length and the shift of the gesture signal in time. The suggested system was tested on three gesture classes: circular motion, tapping and other/unknown gesture. Using the hardware module the training and the testing sets were collected. In course of experiments, it was established that the best training results for the selected network architecture and the collected training set are achieved using the Adam optimizer. The experiment with different number of hidden units in the BLSTM layer showed that the highest recognition rate may be achieved when 100 units are used. Particularly, the suggested system with the specified number of hidden units in BLSTM layer was able to attain the recognition error less than 3.5% (recognition rate of 96.7%) on the test set.

Keywords

#gesture recognition #inertial system #neural networks #BLSTM

References

"World Federation of the Deaf," [Online]. Available: http://wfdeaf.org/faq . [Accessed 22 08 2019].
S. Alashhab, A.-J. Gallego and M. Lozano, "Hand gesture detection with convolutional neural networks," Advances in Intelligent Systems and Computing, p. 45–52, 2018. DOI: 10.1007/978-3-319-94649-8_6.
P. O. Yahanov and I. V. Redko, "Perseptronnyi klasyfikator teplovoho komfortu," Visnyk KNUTD, no. 6 (128), p. 29–38, 2018. DOI: 10.30857/1813-6796.2018.6.3.
J. Tompson, M. Stein, Y. LeCun and K. Perlin, "Real-time continuous pose recovery of human hands using convolutional networks," ACM Transactions on Graphics (ToG), vol. 33, no. 5, p. 169–173, 2014. DOI: 10.1145/2629500.
S. Ji, W. Xu, M. Yang and K. Yu, "3D convolutional neural networks for human action recognition," IEEE transactions on pattern analysis and machine intelligence (TPAMI), vol. 35, no. 1, p. 221–231, 2013. DOI: 10.1109/TPAMI.2012.59.
N. Nishida and H. Nakayama, "Multimodal gesture recognition using multi-stream recurrent neural network," Image and Video Technology PSIVT 2015. Lecture Notes pn Computer Science, vol. 9431, p. 682–694, 2015. DOI: 10.1007/978-3-319-29451-3_54.
N. Neverova, C. Wolf, G. W. Taylor and F. Nebout, "Multi-scale deep learning for gesture detection and localization," Computer Vision – ECCV 2014 Workshops. ECCV. Lecture Notes in Computer Science, vol. 8925, p. 474–490, 2014. DOI: 10.1007/978-3-319-16178-5_33.
R. V. Siriak and I. S. Skarha-Bandurova, "Model obrobky potokovykh danykh dlia rozpiznavannia okremykh odynyts zhestovoi movy," Visnyk Natsionalnoho tekhnichnoho universytetu "KhPI", vol. 42, no. 1318, p. 73–81, 2018.
"Spectra Symbol Flex Sensor," [Online]. Available: https://www.spectrasymbol.com/product/flex-sensors/ .
A. Graves and J. Schmidhuber, "Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures," Neural Networks, vol. 18, no. 5-6, p. 602–610, 2005. DOI: 10.1016/j.neunet.2005.06.042.
I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, Massachusetts: MIT Press, 2016. ISBN: 978-0262035613.
G. Lefebvre, S. Berlemont, F. Mamalet and C. Garcia, "Inertial Gesture Recognition with BLSTM-RNN," in Artificial Neural Networksю Methods and Applications in Bio-/Neuroinformatics, P. Koprinkova-Hristova, V. Mladenov and K. N. K., Eds., Springer International Publishing Switzerland, 2015, pp. 393–410. DOI: 10.1007/978-3-319-09903-3_19.
D. P. Kingma and J. L. Ba, "Adam: A Method for Stochastic Optimization," in International Conference on Learning Representations (ICLR), San Diego, 2015. URL: https://arxiv.org/pdf/1412.6980.pdf .