Computer Vision & CG

Break down the border between virtual and real

Real World Sensing

These sensing technologies understand the 3D real world around our users and devices by determining their position, orientation, and surrounding distances, then integrating the results of multiple observations. We are working on 3D computer vision technologies such as depth estimation, visual SLAM, and 3D modeling algorithms in cameras. These technologies can potentially be utilized in a broad range of Sony business areas from mobile and gaming AR to robot navigation. Our goal is to achieve the highest level of performance in the world by not only developing algorithms but also linking them tightly to our proprietary image sensors.

Visual SLAM image

Free-Viewpoint Visualization

Free-viewpoint video technology captures the real-world as 3D data. It enables viewing of the video from any desired viewpoint. This technology is comprised of two capturing methods: omnidirectional (inside-out) visualization and arbitrary direction free-viewpoint (outside-in) visualization. Currently omnidirectional visualization has 3 degrees of freedom, however, we are developing omnidirectional visualization with 6 degrees of freedom which integrates 3 degrees of translation freedom, as well as volumetric capture which captures specific areas of space to achieve arbitrary direction free-viewpoint visualization. We are also working on technological development utilizing video and imaging technology we have accumulated so far to create photorealistic expressions that appear to be real photographed content despite actually being computer graphics.

from shooting to playback of arbitrary direction free-viewpoint visualization

Image Recognition

We develop image recognition technologies for various Sony products such as robotics, cameras, games and mobile products. Specific image recognition technologies we are developing include face recognition, gesture recognition, object recognition and semantic segmentation. These are achieved through machine learning technologies such as deep learning. With our research activities on the various recognition capabilities needed for synergies between AI and robotics and other such requirements, we will deliver potent recognition devices which run in real time in the real world to be utilized in a wide range of Sony products and services.

information extraction flow in face recognition

Computational Photography

Recently a lot of attention has been focused on the technological field of computational photography, which creates new features by changing and controlling special materials and characteristics of imaging systems (optical systems, lighting and sensors). We combined Sony’s proprietary imaging signal processing technology with our original polarization and multispectral image sensors to offer a variety of new features such as highly accurate acquisition of shape data and measuring the activation levels of shrubs and vegetation.

flow of depth estimation by polarization imaging

Page Top