Project Title
SignSight Every Gesture Clearly Seen
Description
The first round of design focused on developing a 3D product, specifically a camera accessory based on MediaPipe. This camera serves as a supplementary device, while the core product is a digital design—a redesigned language learning app. The app is a refined and optimized version of an existing product, aimed at enhancing the sign language learning experience. The primary goal is to provide a more effective learning pathway for sign language learners, ultimately facilitating better communication between Deaf and hearing individuals.
s a high-precision device powered by computer vision and AI, designed for sign language learning, communication assistance, and language translation. It accurately captures hand gestures, movement trajectories, hand orientation, facial expressions, and body posture in real time, ensuring precise sign language recognition.
Key features include:
- Multimodal Recognition: Integrates hand shape, motion, positioning, and facial expressions to enhance accuracy.
- Semantic Analysis & Conversion: Seamlessly translates sign language into written or spoken language and supports reverse translation for improved communication between the Deaf and hearing communities.
- Adaptive Learning & Optimization: Employs machine learning to continuously refine recognition models, accommodating individual signing styles and variations.
- Visual Feedback & Interaction: Works with AR, projection, or screen-based interfaces to provide intuitive learning experiences and interactive sign language practice.
Process
MediaPipe’s tracking conditions require hand movements and upper body skeletal binding, aligning with the five key components of sign language:
- Handshape – The specific form or shape of the hands during signing.