Hand Movement Tracking

Hand Movement Tracking

Hand Movement Tracking

Hand Movement Tracking


For human beings, gestures have always been a common form of communication, even before the evolution of language. The latest advancements in the fields of computer vision, image processing, and pattern recognition have paved the way for real-time vision-based hand gesture recognition to be feasible

.1. Introduction

In this project, hand gestures captured by a webcam are utilized for intuitive human-computer interaction. A Skin Color Histogram approach is employed, allowing the creation of a segmented hand image to differentiate it from the background. Moreover, the histogram tends to have greater accuracy and takes into account the current light conditions. The result shows that this system can operate well, enabling interaction with a computer using hand gestures and eliminating the need for external sensors.

  1. Tools and Libraries Used:

  1. OpenCV (cv2):

    • Role: Computer vision library.

    • Functions: Video frame capture (cv2.VideoCapture()), color space conversion (cv2.cvtcolor()), morphological operations (cv2.erode(), cv2.dilate()), contour finding (cv2.findContours()).

    • Significance: Crucial for image and video processing tasks.

  2. NumPy:

    • Role: Numerical computing library.

    • Functions: Array manipulation, mathematical operations.

    • Significance: Enhances efficiency in array handling and calculations.

  3. Matplotlib:

    • Role: Plotting library.

    • Functions: Image display (matplotlib.pyplot.imshow()), result visualization.

    • Significance: Facilitates data visualization.

3. Implementation

Firstly, video is captured, and each frame is converted from BGR to the HSV color space using the function c2.cvtcolor() to obtain HSV values. This step is crucial to obtain a mask later. HSV ranges for a specific color are defined, with red chosen in this case.

Morphological operations based on the image's shape, namely Erosion and Dilation, are then performed. Erosion helps "erode" the edges, eliminating some white noise. Dilation, on the other hand, slides around and converts areas to white if the entire area isn't black. For noise removal, erosion is followed by dilation, compensating for the object shrinkage caused by erosion.

The segmentation's downside is that the blue color will be detected, but only the pen is desired. To address this, the "bitwise and" operation is used on the foreground mask and the HSV mask, resulting in the final mask. The cv2.findContours() function is employed to draw a curve joining all continuous points along the boundary, having the same color or intensity (using cv.CHAIN_APPROX_SIMPLE). Finally, everything is displayed.

At the end, the functions cap.release() and cv2.destroyAllWindows() allow the release of the webcam and the closure of all imshow() windows.

3. Implementation

Firstly, video is captured, and each frame is converted from BGR to the HSV color space using the function c2.cvtcolor() to obtain HSV values. This step is crucial to obtain a mask later. HSV ranges for a specific color are defined, with red chosen in this case.

Morphological operations based on the image's shape, namely Erosion and Dilation, are then performed. Erosion helps "erode" the edges, eliminating some white noise. Dilation, on the other hand, slides around and converts areas to white if the entire area isn't black. For noise removal, erosion is followed by dilation, compensating for the object shrinkage caused by erosion.

The segmentation's downside is that the blue color will be detected, but only the pen is desired. To address this, the "bitwise and" operation is used on the foreground mask and the HSV mask, resulting in the final mask. The cv2.findContours() function is employed to draw a curve joining all continuous points along the boundary, having the same color or intensity (using cv.CHAIN_APPROX_SIMPLE). Finally, everything is displayed.

At the end, the functions cap.release() and cv2.destroyAllWindows() allow the release of the webcam and the closure of all imshow() windows.

3. Implementation

Firstly, video is captured, and each frame is converted from BGR to the HSV color space using the function c2.cvtcolor() to obtain HSV values. This step is crucial to obtain a mask later. HSV ranges for a specific color are defined, with red chosen in this case.

Morphological operations based on the image's shape, namely Erosion and Dilation, are then performed. Erosion helps "erode" the edges, eliminating some white noise. Dilation, on the other hand, slides around and converts areas to white if the entire area isn't black. For noise removal, erosion is followed by dilation, compensating for the object shrinkage caused by erosion.

The segmentation's downside is that the blue color will be detected, but only the pen is desired. To address this, the "bitwise and" operation is used on the foreground mask and the HSV mask, resulting in the final mask. The cv2.findContours() function is employed to draw a curve joining all continuous points along the boundary, having the same color or intensity (using cv.CHAIN_APPROX_SIMPLE). Finally, everything is displayed.

At the end, the functions cap.release() and cv2.destroyAllWindows() allow the release of the webcam and the closure of all imshow() windows.

3. Implementation

Firstly, video is captured, and each frame is converted from BGR to the HSV color space using the function c2.cvtcolor() to obtain HSV values. This step is crucial to obtain a mask later. HSV ranges for a specific color are defined, with red chosen in this case.

Morphological operations based on the image's shape, namely Erosion and Dilation, are then performed. Erosion helps "erode" the edges, eliminating some white noise. Dilation, on the other hand, slides around and converts areas to white if the entire area isn't black. For noise removal, erosion is followed by dilation, compensating for the object shrinkage caused by erosion.

The segmentation's downside is that the blue color will be detected, but only the pen is desired. To address this, the "bitwise and" operation is used on the foreground mask and the HSV mask, resulting in the final mask. The cv2.findContours() function is employed to draw a curve joining all continuous points along the boundary, having the same color or intensity (using cv.CHAIN_APPROX_SIMPLE). Finally, everything is displayed.

At the end, the functions cap.release() and cv2.destroyAllWindows() allow the release of the webcam and the closure of all imshow() windows.

4. Challenges

One of the challenges faced in detecting fingers was differentiating a hand from the background and identifying the tip of a finger.

designed by yaba