Abstract :

 GesturePlus is a comprehensive human- computer interaction system that integrates hand gestures, voice commands, and a chatbot for seamless handling. Through the use of computer vision and machine learning, GesturePlus recognizes hand gestures through image pre- processing, fingertip detection, and real-time classification with high accuracy. GesturePlus finds great use within sterile environments or for people with limited mobility in an intuitive manner that replaces the traditional input devices.

---

Keywords :

Chatbot, Computer Vision, Gesture Recognition, Human-Computer Interaction (HCI), Machine learning, Multi- modal Interaction, Retrieval-Augmented Generation (RAG), Voice Command.

---

References :

1. D.-S. Tran, N.-H. Ho, H.-J. Yang, S.-H. Kim, and G. S. Lee, “Real-time virtual mouse system using RGB-D images and fingertip detection,” Multimedia Tools and Applications, vol.  80,  no.  7,  pp.  10473–10490, Nov. 2020. doi: https://doi.org/10.1007/s11042-020-10156-5. Available: http://sclab.jnu.ac.kr/wp-content/uploads/2021/03/ Tran2021Article Real-timeVirtualMouseSystemUsi.pdf.
2. L. Guo, Z. Lu, and L. Yao, “Human-Machine Interaction Sensing Technology Based on Hand Gesture Recognition: a Review,” IEEE Transactions on Human-Machine Systems, vol. 51, no. 4,pp. 300–309, Aug. 2021. doi: https://doi.org/10.1109/thms.2021.3086003.
3. M. J. Vidya, S. Vineela, P. Sathish, and A. S. Reddy, “Gesture- Based Control of Presentation Slides using OpenCV,” IEEE, Aug. 2023. doi: https://doi.org/10.1109/icaiss58487.2023.10250520.
4. B. Kohli, T. Choudhury, S. Sharma, and P. Kumar, “A Platform for Human-Chatbot Interaction Using Python,” IEEE  Xplore,  Aug.  01,  2018.  doi:  https:\//doi.org/10.1109/ICGCIoT.2018.8753031. Available: https://ieeexplore.ieee.org/abstract/document/8753031. [Accessed: Jun. 21, 2021].
5. R. Dudhapachare, M. Awatade, P. Kakde, N. Vaidya, M. Kapgate, and R. Nakhate, “Voice Guided, Gesture Controlled Virtual Mouse,” IEEE Xplore, May 2023. doi: https://doi.org/10.1109/ incet57972.2023.10170317.
6. M. Oudah, A. Al-Naji, and J. Chahl, “Hand Gesture Recognition Based on Computer Vision: A Review of Techniques,” Journal of Imaging, vol. 6, no. 8, p. 73, Jul. 2020. doi: https://doi.org/10. 3390/jimaging6080073.
7. D. Sarma and M. K. Bhuyan, “Methods, Databases and Recent Advancement of Vision-Based Hand Gesture Recognition for HCI Systems: a Review,” SN Computer Science, vol. 2, no. 6, Aug. 2021. doi: https://doi.org/10.1007/s42979-021-00827-x.
8. E. Dinan, S. Roller, K. Shuster, A. Fan, M. Auli, and J. Weston, “Wizard of Wikipedia: Knowledge-Powered Conver- sational agents,” arXiv:1811.01241 [cs], Feb. 2019. Available: https://arxiv.org/abs/1811.01241.
9. C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja,M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee, W.-T. Chang, W. Hua, M. Georg, and M. Grundmann, ”MediaPipe: A Framework for Building Perception Pipelines,” arXiv preprint arXiv:1906.08172, 2019.
10. A. Dhakad and S. Singh, ”Python-Powered Speech-to-Text: A Comprehensive Survey and Performance Analysis,” International Journal of Engineering Research & Technology (IJERT), vol. 12, no. 09, Sep. 2023.