[vc_empty_space][vc_empty_space]
Integration of Indonesian Speech and Hand Gesture Recognition for Controlling Humanoid Robot
Fakhrurroja H.a, Riyantoa, Purwarianti A.a, Prihatmanto A.S.a, Machbub C.a
a School of Electrical Engineering Informatics, Bandung Institute Technology, Indonesia
[vc_row][vc_column][vc_row_inner][vc_column_inner][vc_separator css=”.vc_custom_1624529070653{padding-top: 30px !important;padding-bottom: 30px !important;}”][/vc_column_inner][/vc_row_inner][vc_row_inner layout=”boxed”][vc_column_inner width=”3/4″ css=”.vc_custom_1624695412187{border-right-width: 1px !important;border-right-color: #dddddd !important;border-right-style: solid !important;border-radius: 1px !important;}”][vc_empty_space][megatron_heading title=”Abstract” size=”size-sm” text_align=”text-left”][vc_column_text]© 2018 IEEE.This paper proposes the integration method of Indonesian speech and hand gesture recognition for controlling humanoid robot. A Microsoft Kinect v2 is used to recognize human speech and gesture so that the humanoid robot can follow the command. The recognition of speech grammar is done by creating Indonesian language model which is converted from a combination of English lexicon-grammar. The method of gesture recognition uses Support Vector Machine (SVM) with Directed Acyclic Graph (DAG) decision. Then, the result is obtained from a combined score for all speech and gesture recognition using dynamic programming. Compare to the speech or gesture recognition only, the experimental results prove that the integration of Indonesian speech and gesture recognition has a better accuracy. The speech and gesture commands being tested are forward, backward, turn left, turn right, stop, sit down, and stand up. The testing was conducted by six different people with different gender and age where the distance between the Kinect and the human is around 1,5 meters. The rates of average accuracy of speech recognition, gesture recognition, and integration of speech and gesture are 80.71%, 92.74%, 94.05%, respectively.[/vc_column_text][vc_empty_space][vc_separator css=”.vc_custom_1624528584150{padding-top: 25px !important;padding-bottom: 25px !important;}”][vc_empty_space][megatron_heading title=”Author keywords” size=”size-sm” text_align=”text-left”][vc_column_text]Directed acyclic graph (DAG),Gesture commands,Hand-gesture recognition,Humanoid robot,Indonesian languages,Integration method,Lexicon-grammar,Microsoft kinect[/vc_column_text][vc_empty_space][vc_separator css=”.vc_custom_1624528584150{padding-top: 25px !important;padding-bottom: 25px !important;}”][vc_empty_space][megatron_heading title=”Indexed keywords” size=”size-sm” text_align=”text-left”][vc_column_text][/vc_column_text][vc_empty_space][vc_separator css=”.vc_custom_1624528584150{padding-top: 25px !important;padding-bottom: 25px !important;}”][vc_empty_space][megatron_heading title=”Funding details” size=”size-sm” text_align=”text-left”][vc_column_text]ACKNOWLEDGMENT The first author acknowledges support from the Indonesia Endowment Fund for Education (LPDP –Lembaga Pengelola Dana Pendidikan) scholarship, Ministry of Finance. This research was supported by Program of Post Graduate Team Research 2018 (Penelitian Tim Pasca Sarjana 2018) from The Ministry of Research, Technology and Higher Education, Republic of Indonesia. The authors gratefully acknowledge the anonymous reviewers for their valuable comments.[/vc_column_text][vc_empty_space][vc_separator css=”.vc_custom_1624528584150{padding-top: 25px !important;padding-bottom: 25px !important;}”][vc_empty_space][megatron_heading title=”DOI” size=”size-sm” text_align=”text-left”][vc_column_text]https://doi.org/10.1109/ICARCV.2018.8581071[/vc_column_text][/vc_column_inner][vc_column_inner width=”1/4″][vc_column_text]Widget Plumx[/vc_column_text][/vc_column_inner][/vc_row_inner][/vc_column][/vc_row][vc_row][vc_column][vc_separator css=”.vc_custom_1624528584150{padding-top: 25px !important;padding-bottom: 25px !important;}”][/vc_column][/vc_row]