Covering Disruptive Technology Powering Business in The Digital Age

Home > Archives > News > C-Face: An Ear-Mounted Device That Can Recognise Your Facial Expressions
C-Face: An Ear-Mounted Device That Can Recognise Your Facial Expressions
November 4, 2020 News


Primarily, the usage of earphones or headphones is for listening to music or watching videos. However, researchers from Cornell University had something else in mind when making C-Face, an ear-mounted wearable sensing technology that reads a user’s facial expression and recognises speech for simple mobile commands.

C-Face uses two miniature cameras to continuously reconstruct facial expressions using deep-learning contours of the face, with the purpose of facial expression detection (outputting emojis) and silent speech recognition. This device can help disabled people to express their emotions through emoji, which will not need any front camera on their phones.

Also, C-Face promises better collaboration for the virtual reality environment, as default avatars do not really show what the user is feeling. As for the commands, a user can mouth or cue phrases such as “next song”, “pause” and “volume down” without really speaking, suitable for environments where it should be silent or where there is too much noise. This feature can also work even if a user is wearing a face mask.

The device works by reading the contours of your face – such as eyes, mouth, eyebrows – and then processing such data points to output an emoji or command. “When facial muscles move, the contours of the face change from the point of view of the ear-mounted cameras. These subtle changes are fed into a deep-learning model, which continuously outputs 42 facial feature points representing the shapes and positions of the mouth, eyes and eyebrows. To evaluate C-Face, we embedded our technology into headphones and earphones”, the researchers said.

Since there are still restrictions due to the pandemic, the study was done with only nine participants. However, the researchers still evaluated an average emoji recognition accuracy of 88.6% and 84.7% for the average silent speech word accuracy. In addition, the researchers are also hoping to use a sensing technology that consumes less power as the current device has limited battery capacity.