​Face Gesture​Laptop Control

Overview

It is a Human computer interaction project. Here we have written a code such that if we wink(😉) right eye, right click is done and if we wink left, left click is done. If we move nose, it will move the cursor across the screen. Also if we open our mouth, scrolling mode is enabled and the same can be used to disable the scrolling mode.
​​

Tools Used

  • ubuntu == 18.04
  • opencv-python == 4.1.0
  • numpy==1.16.4
  • dlib==19.17.0
  • matplotlib==3.1.0
  • pyautogui==0.9.4
  • scipy==1.3.0
  • imutils==0.5.2

Features

Without the use of any physical help, the user can now be able to use the mouse using simple facial gestures such as blinking. A face is divided into 68 distinct landmarks.

Exploiting the orientation of the points on the eye/mouth, we use the concept of Eye Aspect Ratio /Mouth Aspect Ratio respectively,  which is used to determine whether the eye/mouth is open or closed.​

For moving the mouse cursor across the screen, the user must move the nose out of a reference circle. In whatever direction the user keeps their nose with respect to the center of the circle, the mouse will move in that direction. Keeping the nose tip in the reference circle will stop the moving cursor.

APPLICATIONS

  1. The model offers two important functions. A detector to detect the face and a predictor to predict the landmarks. Face recognition can be used for crimefighting or security purposes. Click here.
  2. Face unlocking in various devices can be made possible using face recognition.
  3. Controlling the cursor in this specific way might be useful for bedridden people.
  4. The possibility for people with limited mobility of the upper limbs to operate the cursor and hence various tasks increases. They can view videos on youtube read an e-book etc. using our model.

OUR CODE

​You can check our code from this github repository- GITHUB REPOSITORY

Video Demonstration

TEAM MEMBERS: 

  • Akshata Kinage
  • Arihant Gaur
  • Nilakshi Rekhawar
  • Shubhan Rukmangad

MENTORS:

  • Rohit Lal
  • Abhay Khandelwal
  • Shubham Chauhan