Camera and Voice Control-Based Human–Computer Interaction Using Machine Learning

Abstract

Human–computer interaction comes with various hardware implementations out of which is mouse. Latest mouses took their form into wireless with the help of Bluetooth technology or with the help of radio frequencies, which are connected by the receiver, and mouse is powered by batteries. In the proposed method, it uses OpenCV, ML and Python and fingertips controlled and audio cues. These restrictions will be removed by adding a webcam or built-in camera enabling computer vision-based hand movements and fingertip detection. The machine learning algorithm is employed in the system's algorithm, which is the computer can be virtually controlled with hand movements to do various mouse features such as open folder, close folder, refresh, move up, move down along with cursor movements with the help of hand. Deep learning is used to track the hand movements. Hence, the put forward method will be able to make use of the computer's ability to perform tasks such that with the help of audio cues we will be able to command PC and also protect ourselves from infectious diseases such as Covid-19.