Sign to Speech: A Machine Learning Approach for Deaf and Mute Communication

Abstract

This research demonstrates a novel attempt to help people who are both deaf and mute by creating a communication assist system that translates hand signs into words. The system uses a camera to capture hand movements and the trained recognition model identifies them. After recognition, text translation followed by speech synthesis through a voice module is performed. To train and evaluate the system, a custom dataset capturing common gestures was created. The sign-to-speech solution is tailored to operate on constrained, cost-effective hardware such as smartphones and tablets. Furthermore, this review discusses the commonly used datasets in sign-to-speech research and their limitations in terms of size, diversity, and standardization. It also suggests a general flow of implementation starting from data collection, preprocessing, feature extraction, model training, and conversion to speech. The paper highlights key challenges such as gesture variability, occlusion, and real-time processing.

Keywords: machine learning, Sign Language, CNN