Skip to content
Microsystems, Electronics and Acoustics
ISSN 2523-4447 · e‑ISSN 2523-4455 Open Access · CC BY-SA 4.0
Vol. 27 · Issue 2 · 2022 Aug 21, 2022 Acoustical devices and systems

Analysis of the Microcontroller Resources Using Specifics for Speech Recognition

АР
Анна Романівна Рижова Corresponding Національний технічний університет України "Київський політехнічний інститут імені Ігоря Сікорського" Ukraine
ЮО
Юрій Олексійович Оникієнко Національний технічний університет України "Київський політехнічний інститут імені Ігоря Сікорського" Ukraine
Pages265406-1-265406-7 PublishedAug 21, 2022
PDF (Ukrainian)
VOL 27 · ISS 2 · 2022 View issue

Abstract

The use of neural networks for information recognition, in particular, voice, expands the functional capabilities of embedded systems on microcontrollers. But it is necessary to take into account the limitations of the microcontroller resources. The purpose of the work is to analyze the impact of voice processing parameters and neural network architecture on the degree of microcontroller resources usage. To do this, a database of samples of the keyword, samples of other words and voices, and samples of noise are created, the probability of recognizing the keyword among other words and noises is evaluated, the dependence of the amount of memory used on the microcontroller and the decision-making time on the number MFC coefficients is established, the dependence of the amount of used memory of the microcontroller and the decision-making time on the type of convolutional neural network is established also. During the experiment, the Arduino Nano 33 BLE Sense development board was used. The neural network model was built and trained on the Edge Impulse software platform. To conduct the experiment, three groups of data with the names "hello", "unknown", "noise" were created. The group "hello" contains 94 examples of the word "hello" in English, spoken by a female voice. The "unknown" group contains 167 examples of other words pronounced by both female and male voices. The "noise" group contains 166 samples of noise and random sounds. According to Edge Impulse's recommendation, 80% of the samples from each of the data groups were used to train the neural network model, and 20% of the samples were used for testing. Analysis of the results shows that with an increase in the number of MFC coefficients and, accordingly, the accuracy of keyword recognition, the amount of program memory occupied by the code increases by 480 bytes (less than 1%). For the nRF52840 microcontroller, this is not a significant increase. The amount of RAM used during the experiment did not change. Although the calculation time of the accuracy of the code word definition increased by only 14 ms (less than 5%) with the increase in the number of MFC coefficients, the calculation procedure is quite long (approximately 0.3 s) compared to the sound sample length of 1 s. This can be a certain limitation when processing a sound signal with 32-bit microcontrollers. To analyze phrases or sentences, it is necessary to use more powerful microcontrollers or microprocessors. Based on the results of experimental research, it can be stated that the computing resources of 32-bit microcontrollers are quite sufficient for recognizing voice commands with the possibility of pre-digital processing of the sound signal, in particular, the use of low-frequency cepstral coefficients. The selection of the number of coefficients does not significantly affect the amount of used FLASH and RAM memory of the nRF52840 microcontroller. The comparison results show the superiority of the 2D network in the accuracy of the keyword definition for both 12 and 13 MFC coefficients. The use of a one-dimensional convolutional neural network for voice sample recognition in the conducted experiment provides memory savings of approximately 5%. The quality of keyword recognition with the number of MFC coefficients of 12 is approximately 0.7. For 17 MFC coefficients, the recognition quality is already 0.97. The amount of RAM used in the case of the 2D network has decreased slightly. Voice sample processing time for both types of networks is practically the same. Thus, 1D convolutional neural networks have certain advantages in microcontroller applications for voice processing and recognition. The limitation of voice recognition on the microcontroller is the sufficiently long processing time of the sound sample (approximately 0.3 s) with the duration of the sample itself being 1 s, this can be explained by a sufficiently low clock frequency of 64 MHz. Increasing the clock frequency will reduce the calculation time.

Keywords

#microcontrollers #mel-frequency cepstral coefficients #convolutional neural networks #voice recognition

References

  1. S. Misra, T. Das, P. Saha, U. Baruah and R. H. Laskar, "Comparison of MFCC and LPCC for a fixed phrase speaker verification system, time complexity and failure analysis," 2015 International Conference on Circuits, Power and Computing Technologies [ICCPCT-2015], 2015, pp. 1-4, DOI: https://doi.org/10.1109/ICCPCT.2015.7159307 .
  2. Zheng, F., Zhang, G. & Song, Z. “Comparison of different implementations of MFCC”, J. Computer Science & Technology 16, 2001, pp.582–589, DOI: https://doi.org/10.1007/BF02943243
  3. Md Sahidullah, G. Saha, "Design, analysis and experimental evaluation of block based transformation in MFCC computation for speaker recognition," Speech Communication, Volume 54, Issue 4, May 2012, pp. 543-565, DOI: https://doi.org/10.1016/j.specom.2011.11.004
  4. O. Cheng, W. Abdulla and Z. Salcic, "Hardware–Software Codesign of Automatic Speech Recognition System for Embedded Real-Time Applications," in IEEE Transactions on Industrial Electronics, vol. 58, no. 3, pp. 850-859, March 2011, DOI: https://doi.org/10.1109/TIE.2009.2022520
  5. F. Barkani, H. Satori, M. Hamidi, O. Zealouk and N. Laaidi, "Amazigh Speech Recognition Embedded System," 2020 1st Inter-national Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET), 2020, pp. 1-5, DOI: https://doi.org/10.1109/IRASET48871.2020.9092014
  6. A. G. Howard M. Zhu B. Chen D. Kalenichenko W. Wang T. Weyand et al. "Mobilenets: Efficient convolutional neural networks for mobile vision applications" arXiv preprint arXiv 17 Apr 2017, pp. 1-9, DOI: https://doi.org/10.48550/arXiv.1704.04861
  7. D. Sinha and M. El-Sharkawy, "Ultra-thin MobileNet," 2020 10th Annual Computing and Communication Workshop and Con-ference (CCWC), 2020, pp. 0234-0240, DOI: https://doi.org/10.1109/CCWC47524.2020.9031228
  8. Y. -C. Ling, H. -H. Chin, H. -I. Wu and R. -S. Tsay, "Designing A Compact Convolutional Neural Network Processor on Em-bedded FPGAs," 2020 IEEE Global Conference on Artificial Intelligence and Internet of Things (GCAIoT), 2020, pp. 1-7, DOI: https://doi.org/10.1109/GCAIoT51063.2020.9345903
  9. S. M. A. H. Jafri, A. Hemani and L. Intesa, "SPEED: Open-Source Framework to Accelerate Speech Recognition on Embedded GPUs," 2017 Euromicro Conference on Digital System Design (DSD), 2017, pp. 94-101, DOI: https://doi.org/10.1109/DSD.2017.89 .
  10. F. Raffaeli and S. Awad, "Portable low-cost platform for embedded speech analysis and synthesis," 2016 12th International Computer Engineering Conference (ICENCO), 2016, pp. 117-122, DOI: https://doi.org/10.1109/ICENCO.2016.7856455
  11. A. P. Pant, K. -R. Wu and Y. -C. Tseng, "Speak to Action: Offline and Hybrid Language Recognition on Embedded Board for Smart Control System," 2020 International Computer Symposium (ICS), 2020, pp. 85-90, DOI: https://doi.org/10.1109/ICS51289.2020.00026
  12. F. Sutton, R. Da Forno, R. Lim, M. Zimmerling and L. Thiele, "Demonstration abstract: Automatic speech recognition for re-source-constrained embedded systems," IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks, 2014, pp. 323-324, DOI: https://doi.org/10.1109/IPSN.2014.6846784 .
  13. I. Kramberger, M. Grasic and T. Rotovnik, "Door phone embedded system for voice based user identification and verification platform," in IEEE Transactions on Consumer Electronics, vol. 57, no. 3, pp. 1212-1217, August 2011, DOI: https://doi.org/10.1109/TCE.2011.6018876 .
  14. Q. Qu and L. Li, "Realization of embedded speech recognition module based on STM32," 2011 11th International Symposium on Communications & Information Technologies (ISCIT), 2011, pp. 73-77, DOI: https://doi.org/10.1109/ISCIT.2011.6092186 .
  15. “TensorFlow”, TensorFlow.org URL: https://www.tensorflow.org/ (access data 05.06.2022)
  16. "Keras: The Python deep learning API", Keras: the Python deep learning API. URL: https://keras.io/ (access data 04.06.2022).
  17. C. M. J. Galangque and S. A. Guirnaldo, "Speech Recognition Engine using ConvNet for the development of a Voice Command Controller for Fixed Wing Unmanned Aerial Vehicle (UAV)," 2019 12th International Conference on Information & Communi-cation Technology and System (ICTS), 2019, pp. 93-97, DOI: https://doi.org/10.1109/ICTS.2019.8850961 .
  18. J. Dudak, M. Kebisek, G. Gaspar and P. Fabo, "Implementation of machine learning algorithm in embedded devices," 2020 19th International Conference on Mechatronics - Mechatronika (ME), 2020, pp. 1-6, DOI: https://doi.org/10.1109/ME49197.2020.9286705 .
  19. X. Huang, A. Acero, H.-W. Hon, R. Reddy, “Spoken Language Processing - A Guide to Theory, Algorithm, and System Development”, Prentice Hall, 2001, 965pp.
  20. Gu, J., et al., “Recent advances in convolutional neural networks”, Pattern Recognition, 2018, 77: pp. 354-377, DOI: https://doi.org/10.48550/arXiv.1512.07108
  21. S. Kiranyaz, T. Ince, O. Abdeljaber, O. Avci and M. Gabbouj, "1-D Convolutional Neural Networks for Signal Processing Ap-plications," ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 8360-8364, DOI: https://doi.org/10.1109/ICASSP.2019.8682194 .
  22. A. Shenfield, M. Howarth. "A Novel Deep Learning Model for the Detection and Identification of Rolling Element-Bearing Faults" Sensors 2020, 20, 5112. DOI: https://doi.org/10.3390/s20185112
  23. Edge impulse, edgeimpulse.com, URL: https://www.edgeimpulse.com/ (access data 05.06.2022).
  24. Arduino Nano 33 BLE, store.arduino.cc, URL: https://store.arduino.cc/products/arduino-nano-33-ble (access data 05.06.2022).
§ 06 — Related

Similar articles in this journal

Related peer-reviewed studies published in this journal.
View all issues

Experimental Study of the Frequency Characteristics of MEMS Microphones for Railway Noise Monitoring

Vitalii R. Prus, PhD, Assoc. Prof. Oleksii Viktorovych Bogdanov

Abstract–The paper presents the results of an experimental study of the acoustic characteristics of modular MEMS microphones INMP441 and SPH0645, aimed at their…

#acoustic monitoring #MEMS microphone #amplitude-frequency response #directivity pattern
p. 359669.1-359669.6
10.20535/2523-4455.mea.359669

Acoustic Localization of Aerial Objects and Practical Evaluation of its Accuracy

Dr.Sc.(Eng.), Prof. Serhii Anatoliiovych Naida, Dr.Sc.(Eng.), Assoc. Prof. Oleksandr Ivanovych Drozdenko, PhD, Assoc. Prof. Pavlo Vasyliovych Popovych, канд. техн. наук, доц. Катерина Сергіївна Дрозденко, PhD Oleksandr Ihorovych Nyzhnyk, PhD Yaroslav Ivanovych Starovoit

The active use of aerial objects creates a need for passive detection methods. This paper presents an experimental study of an automated acoustic hardware and software…

#acoustic signal processing #sound source localization #microphone arrays #beamforming
p. 359830.1-359830.7
10.20535/2523-4455.mea.359830

Similar Articles

1-10 of 59

You may also start an advanced similarity search for this article.