Sign Language Interpreter

Capstone Project

Diagram of process

Technologies Used









#Google Compute Engine






Build a service that can translate entire ASL phrases into English sentences


  1. Build Machine Learning Model that can correctly guess signing of an ASL word
  2. Build Data processing pipeline that can split a video into x amount of videos based on number of words signed
  3. Extract poses of individual signing
  4. Run Machine Learning Model on each video to get a string of words
  5. Convert words to proper english

What I contributed:

  • Built website for uploading and testing
  • API for video uploads and Interpretation
  • Data Processing
  • Setup everything for Google Cloud
  • Record training data
  • Assist Developing model for training
  • Assist with improving Model accuracy and Model development
  • Produce testing data
  • Produce Video that was presented to class covering entire project


We were able to train a model on ~15 words and given five phrases, correctly interpret and output correct english format!