https://docs.google.com/presentation/d/1ndBKcF274HDTZoVcNW2e7NTFEpLhtSlwp6TzOhnj5jA/edit#slide=id.g20e2308b4e8_0_0
An interactive project invites users to create hand gestures in real-time. It blends the art of hand shadow with digital technology to display lifelike animals on screen.
My interactive project combines the traditional art of hand shadow with modern digital technology, inviting users to create real-time hand gestures. Using ml5.js's Handpose and Body Segmentation models, the system transforms these gestures into lifelike animal animations on screen.
Drawing inspiration from the art of hand shadow—where simple hand movements create vivid, imaginative forms—this project adds a digital dimension to the tradition. The system recognizes specific hand positions and converts them into animated animals: a "Yeah" gesture becomes a rabbit, while a "Rock" gesture transforms into a cat. Advanced segmentation technology seamlessly blends the user's movements with digital animations, creating a clean, focused display.
A webcam captures the user's movements in real-time. Handpose tracks finger and detects key points on the user's hand, while Body Segmentation separates the user from their background for crystal-clear visuals. Together, these technologies create an enchanting experience where users craft living creatures with their hands.
The project aims to connect traditional artistry with modern technology, creating an engaging experience that's both playful and educational. Perfect for all ages, it demonstrates how cultural traditions can evolve through machine learning, inviting everyone to explore where art meets technology.
Step 1: Teachable Machine
I first used a Teachable Machine to train two hand gestures(Butterfly and Dog) as a test.