SONY

close
  1. Home
  2. About mocopi

About mocopi

What is Mobile Motion Capture?

Motion capture is a technique which digitizes the movements of a real person or object and imports them into a computer.
This allows you to reproduce more lifelike, humanly movements with a computer-generated character in video production.
This technique is also being widely used in movies, animations, and game contents within the Sony Group.

Typical motion capture requires studio facilities to install many cameras as well as a tight full-body suit worn by the actor with many markers attached to the body.
In contrast, we have realized a new technology that enables motion capture using only small, lightweight sensors. We call this "Mobile Motion Capture."
With this technology, you can readily digitize a person's movements while wearing everyday clothes whether they are indoors, outdoors, or anywhere and apply those movements to a computer-generated character.

Main Features

With Sony's unique motion capture technology, you can estimate the position and posture of every joint in the body using data from six sensors attached to the head, waist, hands, and legs.

(1) Accelerometers and gyro sensors
(2) Joint posture
(3) Joint position

  1. The three-dimensional position and posture are calculated by detecting the acceleration and angular velocity caused by a person's movements.
  2. AI is used to train the system in advance on various human movements and realize full-body estimation including intermediate joints such as the elbows and knees, where sensors are not attached.

Motion Capture Mechanism

This section explains the mechanism of estimating the three-dimensional position and posture of the entire body of a person with a small number of sensors.
The Mobile Motion Capture technology is realized through the effective use of motion sensing and AI.

STEP 1: Estimate the joint positions which are the reference points of the sensor-attached hands and legs, etc.

  1. Digitize human movement with accelerometers and gyro sensors
  2. Integral calculation
    Generally, a three-dimensional position is found by performing an integral calculation using the acceleration data. However, using only that method has a problem that position errors will accumulate.
  3. Estimate the joint positions using an AI model
    Sony's unique AI model, directly estimates the joint positions and corrects the position errors.
  4. Estimate the reference joints

STEP 2: Estimate the intermediate joint positions such as the elbows and knees where sensor are not attached

  1. Geometric calculation of the positions and posture of the intermediate joints
    There is a problem that the position and posture of an intermediate joint which connects two specific locations where sensors are attached cannot be uniquely determined by a simple geometric calculation. This is because the structure of the human body is complicated, and the joints have a high degree of freedom.
  2. Estimate the joint positions using an AI model
    A natural interpolation of the joint positions is performed by an AI model trained on various human movement.
  3. Estimate the joint positions for the entire body