A downloadable app for Windows

What is it?

Sublynk is a mixed reality Quest3 app that displays AR face tracked English subtitles and an original motion captured database of SEE (Signed Exact English) to encourage learning of sign.

This project was made in two-ish days for MIT Reality Hack 2024. Currently there are only a couple test phrases such as: "Where is the bathroom", "I need help", "Say hello to my little friend", "I love you", "This is delicious". It's more of an open-source proof of concept, but we encourage folks to tinker.


Sublynk is licensed under the MIT open-source license (see PDF file 'license' in project).

Team Members:


     Steven Harmon @stevenharmongames - Programmer

Animation Pipeline / Usability

     Vivian Ngiam @Vivianngiam - Designer 

     Xiaohan Qiu @raychiuxh - Designer

Production & Research

     Mahsa Goodarzi @mgoodarzi - Project Manager

     Steven Bueno @buenosteven02 - Storyteller


Sublynk Win v3.zip 54 MB
Sublynkv4.zip 54 MB
TABLE_88.zip 1 GB

Install instructions


The build export is windows only for now (remnants of commented out Meta Speech SDK for standalone builds and should require no setup aside from ensuring speech input is enabled)

Make sure in your Privacy settings -> Inking & typing personalization that "Getting to know you" is toggled on. The game utilizes Windows's dictation feature.

Hardware Required

Quest 2 or 3 (only tested on 3)

Ensure you have an external webcam plugged in. (The experience uses a ducktaped webcam ontop Quest3 since passthrough is blocked for security reasons) Technically, the external webcam is in addition to the built-in webcam. So you need two webcams for it to take the input of webcam 2. It's hardcoded currently. Sorry!

Software Dependencies

This project is created using Unity 2022.3.18f1, Microsoft's Speech API, MediaPipe's BlazeFace model, and the Leap2 Motion Controller (for mocap of the signs)


Download Sublynk.zip (to be added upon completion)

Extract & Enjoy!

For the standalone executable in the zip, no dependencies needed. Just make sure to calibrate the face tracking by holding the left/right arrow keys to align the hands so that they are properly oriented if they look askew. (this will be fixed later)

If the program is using the wrong webcam, try the other build version v3/v4

Alternatively open project source in Unity and take a look under the hood and build for yourself (regular export process for Unity, one click build)


Log in with itch.io to leave a comment.

Oh my god this is insanely cool!