Jiewen Wang

DuoHands

DuoHands is a web application that enables touch-free remote handshaking. DuoHands comprises of a client application that captures user’s hand poses with a webcam, and a cloud server that connects people together in real time. Users are afforded touch-free social interaction with their hands projected in the virtual space. The virtual hands are rendered as stylized voxels to augment the perception of physical contact and induce synesthesia-like experience.

I started this project as a response to COVID-19 pandemic, aiming to promote anonymous social interaction in a pseudo-tangible form. During the pandemic, while strict enforcement of physical isolation helps to prevent the virus spread, social isolation becomes a pervasive problem that could affect millions of people who are confined at their homes and remain minimal social interaction. Most day-to-day activities such as buying groceries, having chats and meetings became online via Internet to minimize physical contact, it was still impossible to greet with hands such as shaking hands and doing high-fives in a remote manner. There needed to be a digital substitute for handshaking, and more broadly speaking, hand interaction.

Modern machine learning technologies (e.g. Tensorflow.js) and 3D rendering frameworks (e.g. Three.js) have made it possible to build pretty sophisticated computer vision and graphics application in a web browser, eliminating the necessity to install any software or hardware to support real-time hand detection. DuoHands is designed to be compatible with any personal computing devices including smart-phones, laptops and PCs. People under home quarantine can interact with their friends or arbitrary user over the Internet in a playful manner which simulates physical touch.

There are 4 major contributions of this invention:

Alleviating the mental stress induced by home quarantine

Improving social connection via virtual telepresence despite physical distancing

Educating people the importance of not shaking hands physically, thus improving quarantine efficiency

Envisioning a simple, accessible, low-cost and low-latency technology for touch-free hand interaction

Tags

#SocialGood #VirtualTelepresence

PLAY HERE (Note: this is a single-player version. Frame rate could be very low on some devices. Online version is under development.)