Human Interaction

Design the ultimate harmony of system and human beings

Immersive Interaction

We are advancing the VR and projection mapping experience to create immersive interaction that puts users inside extraordinary worlds of entertainment. Utilizing high-speed vision sensors, low-latency projection with prediction algorithms, spatial audio technology and haptics technology with broadband/high dynamic range actuators, we can offer sensory realities that incorporate vision, hearing, and touch, and interactivity to synchronize human motion in real-time. Combining motion sensing technology and deep learning, we are establishing casual motion capture technology which can be utilized anywhere. It captures full body motion in real-time and synchronizes the content. These technologies have already gone into use as new amusement experiences worldwide, integrated with Sony’s movie content and other media. Our future goal is to develop immersive experiences that are even more sophisticated.

the world of immersive interaction

Natural User Interface

Leveraging the human senses and physical characteristics, our aim is to produce natural and comfortable interaction that enables users to complete their intended action without being aware that they are in control. Examples of work in development include gaze interaction that offers natural hands-free control, speech interaction with properties based upon the characteristics of human speech, and motion gesture which detects users’ natural motions with sensors to control devices. We also have behavior recognition interactions which recognizes user behavior even if it is unconscious, and reflects it in device function. Utilizing technology related to the five senses, we are also developing accessibility technology to make operations easier for users with disabilities.

images of gaze interaction and speech interaction and motion gesture

Human Sensing

We are aiming to create systems with awareness of context, which understand a user’s positioning and status and continuously provide various types of assistance. The keys to this are highly accurate recognition through sensor fusion and low power consumption which can be useful in any life scenario. We are compactly implementing the algorithms that utilize Deep Learning within Sony’s original sensing processors, and even within small and light wearable devices. We are also developing biometric authentication technology that can adapt to different contexts with ease and convenience.

image of human sensing

Page Top