ARCHIVES
Research Article
Empowering Nonverbal Communication: A Comprehensive Examination of Sign Language Recognition Systems and Implementation of Interpretation
Krishna1
Navya2
Magan Lodhi3
Khushi Jain4
1234 Department of Information Technology, Dr. A.P.J. Abdul Kalam Technical University, Uttar Pradesh, India.
Published Online: March-April 2024
Pages: 218-230
Cite this article
↗ https://www.doi.org/10.59256/ijire.20240502029
References
[1] World Health Organization. WHO: 1 in 4 people projected to have hearing problems by 2050; 1-Dec-2021. https://www.who.int/news/item/02-03-2021-who-1-in-4-people-projected-to-have-hearing-problems-by-2050
[2] Sign Language. https://education.nationalgeographic.org/resource/sign-language/
[3] Which Countries Recognize Sign Language As An Official Language? https://www.worldatlas.com/articles/which-countries-recognize-sign-languageas-an-official language.html#:~:text=In%20addition%2C%20International%20Sign%20Language,language%20as%20an%20official%20language
[4] Speech disorder. https://en.m.wikipedia.org/wiki/Speech_disorder#:~:text=Developmental%20verbal%20dyspraxia%20also%20known,surgical%20accident%2C%20or%2 0cerebral%20palsy
[5] Nimratveer Kaur Bahia and Rajneesh Rani. 2023. Multi-level Taxonomy Review for Sign Language Recognition: Emphasis on Indian Sign Language. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 22, 1, Article 23 (January 2023), 39 pages. https://doi.org/10.1145/3530259
[6] Sign Language in the Interface: Access for Deaf Signers
https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://huenerfauth.ist.rit.edu/pubs/huenerfauth-hansonchapter38.pdf&ved=2ahUKEwjc5LrWqaOFAxXlyzgGHUjGDYUQFnoECA8QBg&usg=AOvVaw237PGPR6RDGsfjFJYJWgiL
[7] Razieh Rastgoo, Kourosh Kiani, Sergio Escalera, Sign Language Recognition: A Deep Survey, Expert Systems with Applications, Volume 164, 2021, 113794, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2020.113794.
[8] Advances, Challenges, and Opportunities in Continuous Sign Language Recognition.
https://fci.stafpu.bu.edu.eg/Computer%20Science/1273/publications/nada%20bahaa%20ibrahim%20ahmed_1205-1227.pdf
[9] Non-verbal Communication in Depression, By Heiner Ellgring. page No. - 65-66, 126, 131, 140, 165.
https://books.google.co.in/books?hl=en&lr=&id=1ktcau92m8QC&oi=fnd&pg=PR15&dq=Psychological+Effects+of+Sign+Language+Recognition+Syste
ms+on+Non-Verbal+Communicators&ots=UF6YhX8Sc_&sig=vlNCFeGSBv0IznvclWNUSVewK_s&redir_esc=y#v=onepage&q&f=false
[10] Farooq, U., Rahim, M.S.M., Sabir, N. et al. Advances in machine translation for sign language: approaches, limitations, and challenges. Neural Comput & Applic 33, 14357–14399 (2021). https://doi.org/10.1007/s00521-021-06079-3
[11] B. Joksimoski et al., "Technological Solutions for Sign Language Recognition: A Scoping Review of Research Trends, Challenges, and Opportunities," in IEEE Access, vol. 10, pp. 40979-40998, 2022, doi: 10.1109/ACCESS.2022.3161440.
[12] G. G. Nath and C. S. Arun, "Real time sign language interpreter," 2017 IEEE International Conference on Electrical, Instrumentation and
Communication Engineering (ICEICE), Karur, India, 2017, pp. 1-5, doi: 10.1109/ICEICE.2017.8191869.
[13] S. S Kumar, T. Wangyal, V. Saboo and R. Srinath, "Time Series Neural Networks for Real Time Sign Language Translation," 2018 17th IEEE
International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA, 2018, pp. 243-248, doi: 10.1109/ICMLA.2018.00043.
[14] "Real Time Indian Sign Language Detection System." International Journal of Advanced Research in Science, Communication and Technology, null
(2023).:46-52. doi: 10.48175/ijarsct-9540
[15] Design Challenges in Effective Algorithm Development of Sign Language Recognition System. https://www.ijeat.org/portfolio-item/c40300212323/
[16] S. Radhakrishnan, N. C. Mohan, M. Varma, J. Varma and S. N. Pai, "Cross Transferring Activity Recognition to Word Level Sign Language
Detection," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), New Orleans, LA, USA, 2022, pp. 2445-
2452, doi: 10.1109/CVPRW56347.2022.00273.
[17] S. Salim, M. M. A. Jamil, R. Ambar, R. Roslan and M. G. Kamardan, "Sign Language Digit Detection with MediaPipe and Machine Learning
Algorithm," 2022 IEEE 12th International Conference on Control System, Computing and Engineering (ICCSCE), Penang, Malaysia, 2022, pp. 180-184,
doi: 10.1109/ICCSCE54767.2022.9935659.
[18] A Review on Image and Video processing, Byeong-Ho KANG.
https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=ac026b7297ab9409e037bd790fcf6e0318547388
[19] Morikawa C, Kobayashi M, Satoh M, Kuroda Y, Inomata T, Matsuo H, Miura T, Hilaga M. Image and video processing on mobile devices: a survey.
Vis Comput. 2021;37(12):2931-2949. doi: 10.1007/s00371-021-02200-8.
[20] Gesture Voice. https://drive.google.com/file/d/1vbEGzKPnbeCmpsjvq233KtgOxwEPdyrg/view?usp=sharing
[21] Eiichi Asakawa, Naoshi Kaneko, Dai Hasegawa, Shinichi Shirakawa, Evaluation of text-to-gesture generation model using convolutional neural
network, Neural Networks, Volume 151, 2022, Pages 365-375, ISSN 0893-6080, https://doi.org/10.1016/j.neunet.2022.03.041.
[22] K. Shenoy, T. Dastane, V. Rao and D. Vyavaharkar, "Real-time Indian Sign Language (ISL) Recognition," 2018 9th International Conference on
Computing, Communication and Networking Technologies (ICCCNT), Bengaluru, India, 2018, pp. 1-9, doi: 10.1109/ICCCNT.2018.8493808.
[23] Podder KK, Chowdhury MEH, Tahir AM, Mahbub ZB, Khandakar A, Hossain MS, Kadir MA. Bangla Sign Language (BdSL) Alphabets and
Numerals Classification Using a Deep Learning Model. Sensors. 2022; 22(2):574. https://doi.org/10.3390/s22020574
[24] Almasre MA, Al-Nuaim H. Comparison of Four SVM Classifiers Used with Depth Sensors to Recognize Arabic Sign Language Words. Computers.
2017; 6(2):20. https://doi.org/10.3390/computers6020020
[25] "Sign Language Detection." undefined (2022). doi: 10.48550/arxiv.2209.03578
[26] H.R., Akhilesh, Kumar., Mohit, Kumar, Sharma., Rohit., Kunal, Singh, Bisht., Ashish, Kumar., Rachna, Jain., Preeti, Nagrath., Pritpal, Singh. "Sign
language detection and conversion to text using CNN and OpenCV." Nucleation and Atmospheric Aerosols, undefined (2022). doi: 10.1063/5.0108711
[27] Basanta, Mahato. "The Importance and Challenges of Sign Language Translator- A Review." Spectrum of Emerging Sciences, undefined (2023). doi:
10.55878/ses2023-3-1-8
[28] Anif, Hanifa, Setianingrum., Arifa, Fauzia., Dzul, Fadli, Rahman. "Hand-Gesture Detection Using Principal Component Analysis (PCA) and
Adaptive Neuro-Fuzzy Inference System (ANFIS)." Jurnal Teknik Informatika, undefined (2022). doi: 10.15408/jti.v15i1.24869
[29] "Real Time Indian Sign Language Detection System." International Journal of Advanced Research in Science, Communication and Technology,
undefined (2023). doi: 10.48175/ijarsct-9540
[30] Vaidik, Gupta., Rohan, Punjani., Mayur, Vaswani., Jyoti, Kundale. "Video Conferencing with Sign language Detection." undefined (2022). doi: 10.1109/ASIANCON55314.2022.9908973
[31] Kenneth, Mejía-Peréz., Diana-Margarita, Córdova-Esparza., Juan, R., Terven., Ana, M., Herrera-Navarro., Teresa, García, Ramírez., Alfonso,
Ramírez-Pedraza. "Automatic Recognition of Mexican Sign Language Using a Depth Camera and Recurrent Neural Networks." Applied Sciences,
undefined (2022). doi: 10.3390/app12115523
[32] Hezhen, Hu., Wengang, Zhou., Junfu, Pu., Houqiang, Li. "Global-Local Enhancement Network for NMF-Aware Sign Language Recognition."
undefined (2021). doi: 10.1145/3436754
[33] Yugam, Bajaj., Puru, Malhotra. "American Sign Language Identification Using Hand Trackpoint Analysis." arXiv: Computer Vision and Pattern
Recognition, undefined (2022). doi: 10.1007/978-981-16-2594-7_13
[34] S., B., Abdullahi., Kosin, Chamnongthai. "American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven MultiStacked Deep LSTM." Sensors, undefined (2022). doi: 10.3390/s22041406
[35] "Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generation." undefined (2022). doi:
10.48550/arxiv.2207.03714
[36] Yu, Liu., Parma, Nand., Md, Akbar, Hossain., Minh, Nguyen., Wei-Mon, Yan. "Sign language recognition from digital videos using feature pyramid
network with detection transformer." Multimedia Tools and Applications, undefined (2023). doi: 10.1007/s11042-023-14646-0
[37] Rung-Ching, Chen., William, Eric, Manongga., Christine, Dewi. "Recursive Feature Elimination for Improving Learning Points on Hand-Sign
Recognition." Future Internet, undefined (2022). doi: 10.3390/fi14120352
[38] Ozge, Mercanoglu, Sincan., Julio, C., S., Jacques., Sergio, Escalera., Hacer, Yalim, Keles. "ChaLearn LAP Large Scale Signer Independent Isolated
Sign Language Recognition Challenge: Design, Results and Future Research." undefined (2021). doi: 10.1109/CVPRW53098.2021.00386
[39] Stoll, S., Camgoz, N.C., Hadfield, S. et al. Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks. Int J Comput Vis 128, 891–908 (2020). https://doi.org/10.1007/s11263-019-01281-2
[2] Sign Language. https://education.nationalgeographic.org/resource/sign-language/
[3] Which Countries Recognize Sign Language As An Official Language? https://www.worldatlas.com/articles/which-countries-recognize-sign-languageas-an-official language.html#:~:text=In%20addition%2C%20International%20Sign%20Language,language%20as%20an%20official%20language
[4] Speech disorder. https://en.m.wikipedia.org/wiki/Speech_disorder#:~:text=Developmental%20verbal%20dyspraxia%20also%20known,surgical%20accident%2C%20or%2 0cerebral%20palsy
[5] Nimratveer Kaur Bahia and Rajneesh Rani. 2023. Multi-level Taxonomy Review for Sign Language Recognition: Emphasis on Indian Sign Language. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 22, 1, Article 23 (January 2023), 39 pages. https://doi.org/10.1145/3530259
[6] Sign Language in the Interface: Access for Deaf Signers
https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://huenerfauth.ist.rit.edu/pubs/huenerfauth-hansonchapter38.pdf&ved=2ahUKEwjc5LrWqaOFAxXlyzgGHUjGDYUQFnoECA8QBg&usg=AOvVaw237PGPR6RDGsfjFJYJWgiL
[7] Razieh Rastgoo, Kourosh Kiani, Sergio Escalera, Sign Language Recognition: A Deep Survey, Expert Systems with Applications, Volume 164, 2021, 113794, ISSN 0957-4174, https://doi.org/10.1016/j.eswa.2020.113794.
[8] Advances, Challenges, and Opportunities in Continuous Sign Language Recognition.
https://fci.stafpu.bu.edu.eg/Computer%20Science/1273/publications/nada%20bahaa%20ibrahim%20ahmed_1205-1227.pdf
[9] Non-verbal Communication in Depression, By Heiner Ellgring. page No. - 65-66, 126, 131, 140, 165.
https://books.google.co.in/books?hl=en&lr=&id=1ktcau92m8QC&oi=fnd&pg=PR15&dq=Psychological+Effects+of+Sign+Language+Recognition+Syste
ms+on+Non-Verbal+Communicators&ots=UF6YhX8Sc_&sig=vlNCFeGSBv0IznvclWNUSVewK_s&redir_esc=y#v=onepage&q&f=false
[10] Farooq, U., Rahim, M.S.M., Sabir, N. et al. Advances in machine translation for sign language: approaches, limitations, and challenges. Neural Comput & Applic 33, 14357–14399 (2021). https://doi.org/10.1007/s00521-021-06079-3
[11] B. Joksimoski et al., "Technological Solutions for Sign Language Recognition: A Scoping Review of Research Trends, Challenges, and Opportunities," in IEEE Access, vol. 10, pp. 40979-40998, 2022, doi: 10.1109/ACCESS.2022.3161440.
[12] G. G. Nath and C. S. Arun, "Real time sign language interpreter," 2017 IEEE International Conference on Electrical, Instrumentation and
Communication Engineering (ICEICE), Karur, India, 2017, pp. 1-5, doi: 10.1109/ICEICE.2017.8191869.
[13] S. S Kumar, T. Wangyal, V. Saboo and R. Srinath, "Time Series Neural Networks for Real Time Sign Language Translation," 2018 17th IEEE
International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA, 2018, pp. 243-248, doi: 10.1109/ICMLA.2018.00043.
[14] "Real Time Indian Sign Language Detection System." International Journal of Advanced Research in Science, Communication and Technology, null
(2023).:46-52. doi: 10.48175/ijarsct-9540
[15] Design Challenges in Effective Algorithm Development of Sign Language Recognition System. https://www.ijeat.org/portfolio-item/c40300212323/
[16] S. Radhakrishnan, N. C. Mohan, M. Varma, J. Varma and S. N. Pai, "Cross Transferring Activity Recognition to Word Level Sign Language
Detection," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), New Orleans, LA, USA, 2022, pp. 2445-
2452, doi: 10.1109/CVPRW56347.2022.00273.
[17] S. Salim, M. M. A. Jamil, R. Ambar, R. Roslan and M. G. Kamardan, "Sign Language Digit Detection with MediaPipe and Machine Learning
Algorithm," 2022 IEEE 12th International Conference on Control System, Computing and Engineering (ICCSCE), Penang, Malaysia, 2022, pp. 180-184,
doi: 10.1109/ICCSCE54767.2022.9935659.
[18] A Review on Image and Video processing, Byeong-Ho KANG.
https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=ac026b7297ab9409e037bd790fcf6e0318547388
[19] Morikawa C, Kobayashi M, Satoh M, Kuroda Y, Inomata T, Matsuo H, Miura T, Hilaga M. Image and video processing on mobile devices: a survey.
Vis Comput. 2021;37(12):2931-2949. doi: 10.1007/s00371-021-02200-8.
[20] Gesture Voice. https://drive.google.com/file/d/1vbEGzKPnbeCmpsjvq233KtgOxwEPdyrg/view?usp=sharing
[21] Eiichi Asakawa, Naoshi Kaneko, Dai Hasegawa, Shinichi Shirakawa, Evaluation of text-to-gesture generation model using convolutional neural
network, Neural Networks, Volume 151, 2022, Pages 365-375, ISSN 0893-6080, https://doi.org/10.1016/j.neunet.2022.03.041.
[22] K. Shenoy, T. Dastane, V. Rao and D. Vyavaharkar, "Real-time Indian Sign Language (ISL) Recognition," 2018 9th International Conference on
Computing, Communication and Networking Technologies (ICCCNT), Bengaluru, India, 2018, pp. 1-9, doi: 10.1109/ICCCNT.2018.8493808.
[23] Podder KK, Chowdhury MEH, Tahir AM, Mahbub ZB, Khandakar A, Hossain MS, Kadir MA. Bangla Sign Language (BdSL) Alphabets and
Numerals Classification Using a Deep Learning Model. Sensors. 2022; 22(2):574. https://doi.org/10.3390/s22020574
[24] Almasre MA, Al-Nuaim H. Comparison of Four SVM Classifiers Used with Depth Sensors to Recognize Arabic Sign Language Words. Computers.
2017; 6(2):20. https://doi.org/10.3390/computers6020020
[25] "Sign Language Detection." undefined (2022). doi: 10.48550/arxiv.2209.03578
[26] H.R., Akhilesh, Kumar., Mohit, Kumar, Sharma., Rohit., Kunal, Singh, Bisht., Ashish, Kumar., Rachna, Jain., Preeti, Nagrath., Pritpal, Singh. "Sign
language detection and conversion to text using CNN and OpenCV." Nucleation and Atmospheric Aerosols, undefined (2022). doi: 10.1063/5.0108711
[27] Basanta, Mahato. "The Importance and Challenges of Sign Language Translator- A Review." Spectrum of Emerging Sciences, undefined (2023). doi:
10.55878/ses2023-3-1-8
[28] Anif, Hanifa, Setianingrum., Arifa, Fauzia., Dzul, Fadli, Rahman. "Hand-Gesture Detection Using Principal Component Analysis (PCA) and
Adaptive Neuro-Fuzzy Inference System (ANFIS)." Jurnal Teknik Informatika, undefined (2022). doi: 10.15408/jti.v15i1.24869
[29] "Real Time Indian Sign Language Detection System." International Journal of Advanced Research in Science, Communication and Technology,
undefined (2023). doi: 10.48175/ijarsct-9540
[30] Vaidik, Gupta., Rohan, Punjani., Mayur, Vaswani., Jyoti, Kundale. "Video Conferencing with Sign language Detection." undefined (2022). doi: 10.1109/ASIANCON55314.2022.9908973
[31] Kenneth, Mejía-Peréz., Diana-Margarita, Córdova-Esparza., Juan, R., Terven., Ana, M., Herrera-Navarro., Teresa, García, Ramírez., Alfonso,
Ramírez-Pedraza. "Automatic Recognition of Mexican Sign Language Using a Depth Camera and Recurrent Neural Networks." Applied Sciences,
undefined (2022). doi: 10.3390/app12115523
[32] Hezhen, Hu., Wengang, Zhou., Junfu, Pu., Houqiang, Li. "Global-Local Enhancement Network for NMF-Aware Sign Language Recognition."
undefined (2021). doi: 10.1145/3436754
[33] Yugam, Bajaj., Puru, Malhotra. "American Sign Language Identification Using Hand Trackpoint Analysis." arXiv: Computer Vision and Pattern
Recognition, undefined (2022). doi: 10.1007/978-981-16-2594-7_13
[34] S., B., Abdullahi., Kosin, Chamnongthai. "American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven MultiStacked Deep LSTM." Sensors, undefined (2022). doi: 10.3390/s22041406
[35] "Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video Generation." undefined (2022). doi:
10.48550/arxiv.2207.03714
[36] Yu, Liu., Parma, Nand., Md, Akbar, Hossain., Minh, Nguyen., Wei-Mon, Yan. "Sign language recognition from digital videos using feature pyramid
network with detection transformer." Multimedia Tools and Applications, undefined (2023). doi: 10.1007/s11042-023-14646-0
[37] Rung-Ching, Chen., William, Eric, Manongga., Christine, Dewi. "Recursive Feature Elimination for Improving Learning Points on Hand-Sign
Recognition." Future Internet, undefined (2022). doi: 10.3390/fi14120352
[38] Ozge, Mercanoglu, Sincan., Julio, C., S., Jacques., Sergio, Escalera., Hacer, Yalim, Keles. "ChaLearn LAP Large Scale Signer Independent Isolated
Sign Language Recognition Challenge: Design, Results and Future Research." undefined (2021). doi: 10.1109/CVPRW53098.2021.00386
[39] Stoll, S., Camgoz, N.C., Hadfield, S. et al. Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks. Int J Comput Vis 128, 891–908 (2020). https://doi.org/10.1007/s11263-019-01281-2
Related Articles
2024
Embedding Artificial Intelligence for Personal Voice Assistant Using NLP
2024
Analysis of Pedestrian Steel Bridge subjected the Seismic Load and Wind Load using Damper at different Span
2024
Review Paper on Comparison of Asymmetric and Symmetric RCC Building with Soil Structure Interaction by Dynamic Loading
2024
BLYNK RFID and Retinal Lock Access System
2024
ML-Driven Facial Synthesis from Spoken Words Using Conditional GANs
2024
