CharARize
Background and Development Purpose
In face-to-face communication, the appearance and voice quality of a person can make conversations difficult. For instance, it might be challenging to speak smoothly with a cute girl due to nervousness, or it can be hard to speak candidly with an older person who appears stern. In such cases, the person’s appearance can act as noise, distracting from the essence of the communication.
To address this issue, we developed “RealityShiftAR” to transform the visible world and make communication smoother. By using these AR glasses, the appearance of people around you can change to look like animated characters or appear younger or older. Additionally, their voices are modified, enhancing the ease of face-to-face communication. This eliminates the noise caused by appearance, allowing for focused and effective communication.
The inspiration for this project came from Osamu Tezuka’s “Phoenix” where the protagonist sees the surrounding world as monsters. Our goal is to transform reality in a more appealing way and facilitate smoother communication.
System Overview
できたぜ自作シースルーＡＲヘッドセット!!! pic.twitter.com/SerA9ycR6M
— Gakuto Sasabe (@gakuto66) September 20, 2023
-
Capture Video: A camera mounted on a Raspberry Pi captures live video of the surroundings.
-
Face Recognition and Cropping: The captured video is processed using Mediapipe to recognize human faces. The facial regions are then cropped using OpenCV.
-
Image Transmission and Generation: The cropped facial images are sent via WiFi to a connected gaming laptop. The laptop uses Stable Diffusion, accessed through a Web API, to generate transformed images of the faces. These images are then sent back to the Raspberry Pi and stored locally.
-
Overlay and Display: The generated images are overlaid onto the recognized facial regions in the video. This combined video is then displayed on a screen, creating an augmented reality (AR) effect.