Skip to content
KPI Science News
ISSN 2617-5509 · e‑ISSN 2663-7472 Open Access · CC BY 4.0
Vol. 143 · Issue 2 · 2026 Jun 29, 2026 Robotics, Automation and Control Systems

MACHINE LEARNING MODEL FOR RECOGNITION OF UKRAINIAN SIGN LANGUAGE HAND CONFIGURATIONS

KG
Kostiantyn D. Grishyn Corresponding National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute” Ukraine
NN
Nadezhda Nedashkovskaya National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute” Ukraine
Pages7-19 PublishedJun 29, 2026 LicenseOpen Access
PDF
VOL 143 · ISS 2 · 2026 View issue

Abstract

Background. Sign language recognition using Artificial Intelligence (AI) is a relevant and still unsolved problem. For Ukrainian Sign Language (USL), the task is complicated by a lack of publicly available datasets. Consequently, researchers operating under data-constrained conditions should apply additional features for improving model accuracy. Hand configuration is a fundamental characteristic of lexical units in any sign language. In USL 40 configurations exist. The ability to accurately identify configuration would provide relevant features for AI-based sign language recognition systems.

Objective. To develop a machine learning model capable of frame-by-frame recognition of USL hand configurations on a video stream with an accuracy of at least 95% on the test set and an average inference time not exceeding 70 ms.

Methods. As input data for the USL hand configurations recognition models, a system of mathematical indicators, calculated from the keypoints coordinates, extracted via MediaPipe pose estimation system, was proposed. The training dataset was created from camera images of USL hand configurations performed by middle-aged subjects. Following keypoints extraction through MediaPipe, the corresponding vectors of mathematical indicators were computed. Various architectures of random forest and perceptron were evaluated.

Results. The highest performance was achieved by the perceptron with two hidden layers of 300 neurons each, which takes the vector of proposed mathematical indicators as input. The model reached an accuracy of 98.18% on the test set. During testing on a video stream, it identified without error 32 out of 40 USL configurations, and with only sporadic deviations. The average inference latency was 46 ms. Notably, a perceptron of identical architecture trained on raw MediaPipe keypoint coordinates exhibited lower accuracy on a video stream. Furthermore, the proposed indicators improved random forest model performance, confirming their universality.

Conclusion. The optimal model among the considered architectures was a perceptron with two hidden layers of 300 neurons each, which takes the vector of proposed mathematical indicators as input. It identifies in real-time USL hand configurations, achieving an accuracy of 98.18% on the test set. In experiments based on the proposed expert assessment criterion on the video stream, it correctly identified 35% more configurations (80% vs. 45%), compared to the model based on raw keypoint coordinates. Therefore, implementing additional verification based on expert assessment in addition to the standard accuracy metric on the test data is effective. The presented architecture is the first artificial intelligence-based model for the USL hand configuration recognition. Future research will focus on integrating this network into a system for isolated recognition of USL.

Keywords

#Ukrainian Sign Language #machine learning #neural networks #computer vision #gesture recognition #model quality assessment #feature engineering #system analysis

References

  1. "‘Sign Languages are for Everyone!’ UN International Day of Sign Languages." [Online]. Available: https://www.european-agency.org/news/sign-languages-are-everyone-un-international-day-sign-languages . Accessed: Apr. 13, 2026
  2. "UNICEF Ukraine Facebook post February 20, 2025" [Online]. Available: https://www.facebook.com/1789994851793391 . Accessed: Apr. 13, 2026
  3. "SignGPT Building Generative Predictive Transformers for Sign Language. An EPSRC Programme Grant 1/5/2025-30/4/2031" [Online]. Available: https://sites.google.com/view/signgpt/team . Accessed: Apr. 13, 2026
  4. "Google DeepMind SignGemma announcement" [Online]. Available: https://x.com/GoogleDeepMind/status/1927375853551235160 . Accessed: Apr. 13, 2026
  5. "SLAIT at DeafIT Conference about "Breaking communication barriers between Deaf & hearing people" [Online]. Available: https://www.youtube.com/watch?v=UZ1FJP7uO9Y . Accessed: Apr. 13, 2026
  6. "A software that can translate any sign language // Expo Live – KinTrans Inc. – USA" [Online]. Available: https://www.youtube.com/watch?v=29n1YhOVHHA . Accessed: Apr. 13, 2026
  7. "1st place solution – 1DCNN combined with Transformer" [Online]. Available: https://www.kaggle.com/competitions/asl-signs/writeups/hoyeol-sohn-1st-place-solution-1dcnn-combined-with . Accessed: Apr. 13, 2026
  8. S. V. Kulbida, "Konfihuratsiia yak komponent zhestu [Configuration as a sign component]," Osoblyva dytyna : navchannia i vykhovannia. [Special child: teaching and raising], vol. 4, pp. 15–30, 2018. Retrieved from doi: https://doi.org/10.33189/ectu.v4i89.6
  9. N. Camgoz et al., "SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition," in IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017. Retrieved from doi: https://doi.org/10.1109/ICCV.2017.332
  10. O. Koller et al., "Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 9, pp. 2306–2320, 2020. Retrieved from doi: https://doi.org/10.1109/TPAMI.2019.2911077
  11. X. Zhang and K. Duh, "Handshape-Aware Sign Language Recognition: Extended Datasets and Exploration of Handshape-Inclusive Methods," in Findings of the Association for Computational Linguistics: EMNLP, Singapore, pp. 2993–3002, 2023. Retrieved from doi: https://doi.org/10.18653/v1/2023.findings-emnlp.198
  12. A. A. Hosain et al., "FineHand: Learning Hand Shapes for American Sign Language Recognition," in 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), 2020, pp. 700–707. Retrieved from doi: https://doi.org/10.1109/FG47880.2020.00062
  13. A. Carbo and E. Nalisnick, "Improving Handshape Representations for Sign Language Processing: A Graph Neural Network Approach," in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China, pp. 29122–29135. Retrieved from doi: https://doi.org/10.18653/v1/2025.emnlp-main.1483
  14. B. Demircioğlu, G. Bülbül and H. Köse, "Turkish Sign Language recognition with Leap Motion," in 24th Signal Processing and Communication Application Conference (SIU), 2016, pp. 589–592, Retrieved from doi: https://doi.org/10.1109/SIU.2016.7495809
  15. S. Sharma, R. Gupta and A. Kumar, "Continuous sign language recognition using isolated signs data and deep transfer learning," Journal of Ambient Intelligence and Humanized Computing, vol. 14, no. 3, pp. 1531–1542, 2023. Retrieved from doi: https://doi.org/10.1007/s12652-021-03418-z
  16. S. Alyami, H. Luqman and M. Hammoudeh, "Reviewing 25 years of continuous sign language recognition research: Advances, challenges, and prospects," Information Processing & Management, vol. 61, no. 5, 2024. Retrieved from doi: https://doi.org/10.1016/j.ipm.2024.103774
  17. K. Myagila, D.G. Nyambo and M.A. Dida, "Framework for detecting and recognizing sign language using absolute pose estimation difference and deep learning," Machine Learning with Applications, vol. 21, 2025. Retrieved from doi: https://doi.org/10.1016/j.mlwa.2025.100723
  18. R. Rastgoo, K. Kiani and S. Escalera, "Word separation in continuous sign language using isolated signs and post-processing," Expert Systems with Applications, vol. 249, 2024. Retrieved from doi: https://doi.org/10.1016/j.eswa.2024.123695
  19. O. V. Lozynska, "Systema kompiuternoho perekladu ukrainskoi zhestovoi movy z vykorystanniam hramatychno dopovnenoi ontolohii," [System of computer translation for Ukrainian Sign Language using grammatically augmented ontology] Ph.D. dissertation, Lviv Polytechnic National University, Lviv, 2016. [Online]. Available: https://lpnu.ua/sites/default/files/2020/dissertation/1452/dyserlozynska1.pdf . Accessed: Apr. 13, 2026
  20. N.C. Camgöz et al., "Neural Sign Language Translation," IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018. [Online]. Available: https://www-i6.informatik.rwth-aachen.de/~koller/RWTH-PHOENIX-2014-T/ . Accessed: Apr. 13, 2026
  21. H. Zhou et al., "CSL-Daily Dataset", 2021. Available: https://ustc-slr.github.io/datasets/2021_csl_daily/ . Accessed: Apr. 13, 2026
  22. J. Yu, M. Qin, and S. Zhou, "Dynamic gesture recognition based on 2D convolutional neural network and feature fusion," Scientific Reports, vol. 12, 2022. Retrieved from doi: https://doi.org/10.1038/s41598-022-08133-z
  23. Yu. V. Krak et al., "Matematychni metody ta prykladni informatsiini tekhnolohii modeliuvannia, perekladu ta navchannia dlia ukrainskoi zhestovoi movy [Mathematical methods and applied informational technologies of modeling, translation and learning for Ukrainian Sign Language]," Bulletin of Lviv Polytechnic National University: information technologies and networks, no. 854, 2016, pp. 210–227. [Online]. Available: http://nbuv.gov.ua/UJRN/VNULPICM_2016_854_22
  24. A. Hlybovets and M. Bikchentayev, "Recognizing gestures of the ukrainian dactylic alphabet," International Scientific Technical Journal "Problems of Control and Informatics, vol. 68, no. 3, pp. 86–100, 2023. Retrieved from doi: https://doi.org/10.34229/1028-0979-2023-3-9
  25. I. O. Burkhan and O. V. Barabanov, "Rozpiznavannia ukrainskoi zhestovoi movy na osnovi kliuchovykh tochok z vykorystanniam rekurentnykh neironnykh merezh [Keypoint-based Ukrainian Sign Language translation using Recurrent Neural Networks]," Science and Technology Today, vol. 4, no. 18, 2023. Retrieved from doi: https://doi.org/10.52058/2786-6025-2023-4(18)-280-292
  26. N. V. Kuznietsova and S. S. Smirnov, "Uzahalnena metodolohiia rozpiznavannia movy zhestiv na videopotokakh na osnovi neironnykh merezh i transformeriv [A generalized methodology of sign language recognition on video streams based on neural networks and transformers]," Data Recording, Storage & Processing, vol. 25, no. 2. pp. 66–76. 2023. Retrieved from doi: https://doi.org/10.35681/1560-9189.2023.25.2.300527
  27. A. Géron, Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow (2nd ed.), O’Reilly Media Inc., Sebastopol, CA, 2019. [Online]. Available: https://shashwatwork.github.io/assets/files/ml_ebook.pdf . Accessed: Apr. 13, 2026
  28. A. Zhang et al., "Dive into Deep Learning. 5. Multilayer Perceptrons," [Online]. Available: https://www.d2l.ai/chapter_multilayer-perceptrons/index.html . Accessed: Apr. 13, 2026
  29. W. F. Maia et al., "Automatic sign language to text translation using MediaPipe and transformer architectures," Neurocomputing, vol. 642, 2025. Retrieved from doi: https://doi.org/10.1016/j.neucom.2025.130421
  30. "1 Urok Daktylna abetka, kanal «Ukrainska Zhestova mova»" [1 Lesson Dactyl alphabet, channel «Ukrainian Sign Language»]. [Online]. Available: https://www.youtube.com/watch?v=DHo96OMHRyE . Accessed: Apr. 13, 2026
  31. S. S. Kondratiuk, "Ukrainian dactyl alphabet gesture recognition using cross platform software and convolutional neural networks," Artificial Intelligence, vol. 24, pp. 107–113, 2019. Retrieved from doi: https://doi.org/10.15407/jai2019.03-04.107
  32. B. Alsharif et al., "Deep Learning Technology to Recognize American Sign Language Alphabet," Sensors, vol. 23, no. 18, 7970, 2023. Retrieved from doi: https://doi.org/10.3390/s23187970

License

CCBY 4.0
Creative Commons Attribution 4.0 International

This work is openly licensed — share and adapt freely with attribution to the authors and the journal. View license terms ↗

Copyright (c) 2026 Костянтин Д. Грішин, Nadezhda Nedashkovskaya (Автор)

§ 06 — Related

Similar articles in this journal

Related peer-reviewed studies published in this journal.
View all issues

DEEP REINFORCEMENT LEARNING-BASED MAPLESS NAVIGATION ALGORITHM FOR HUSKY ROBOT WITH ADAPTIVE CURRICULUM LEARNING

Dmytro Petrenko, Anatolii Protasov

Issues. Mobile wheeled robots are widely used today in internal logistics environments of various industries, in particular in warehouses, production shops and sorting…

#machine learning with reinforcement #navigation without cartography #mobile robots #control systems
p. 36-43
10.20535/kpisn.2026.2.357072

SIMULATION MODEL OF A DIGITAL TWIN FOR AN NPP POWER UNIT TECHNOLOGICAL PROCESS

Kostiantyn Brovko, Pavlo Budanov, Oleh Velykohorskyi, Yuliia Oliinyk, Nataliia Vynokurova, Serhii Voitenko

Background. Modern nuclear power plant (NPP) units are characterised by high structural complexity, multi-level hierarchical Information and Control (I&C) systems, and a…

#digital twin #power unit #fractal dimension #system-cluster approach
p. 36-43
10.20535/kpisn.2026.2.352429

Most read articles by the same author(s)

Similar Articles

1-10 of 54

You may also start an advanced similarity search for this article.