With 12 tone analysis, metadata can be applied to all songs automatically. The following are some examples of the types of applications that can be implemented using automatically extracted metadata.

• Omakase Channel and SensMe applications which automatically run playlists based on specific musical characteristics (fast, bright, etc.)


• ZAPPIN---A “quick-search” tool which continuously plays back only the chorus portion of songs


• Recommendation features which search for and recommend songs similar to one's favorites


• Automatic creation of slideshows, etc., based on the mood of songs


• Automatic “DJ feature” which bridges together two songs which go well together


• Automatic search for specific intervals of TV or radio programs where music is featured

With 12 tone analysis, music is analyzed through the following processes.

Time-Tone Analysis
The 12 tone analysis process begins with a two-dimensional analysis of the song based on time and tone. There are 12 tones (equivalent to the do-re-mi scale) per octave. When this analysis is performed first, it becomes easier to extract the information needed to carry out subsequent processes, including the detection of the timing and strength of the initial sound, timbre and chord structures.

The filters developed for 12 tone analysis allow rapid high-precision analysis from bass to treble.

Analysis Based on Musical Theory
Using the two-dimensional image obtained through this analysis, a variety of signal processes and detection processes are then carried out to detect features based on musical theory, such as beat elements, including tempo, rhythm and bar, as well as chord progression, key, and song structure.

Previously element technologies, such as chord detection, song structure detection, were treated separately. With 12 tone analysis, all detection processes are integrated, allowing estimation based on the reciprocal use of multiple detection results. This ensures extremely accurate detection processing.

Feature Extraction
The results of time-tone analysis and analyses based on musical theory are next used to extract features that can be used to classify songs. The 12 tone analysis system brought to market by Sony uses several dozen highly independent features to support the classification of music from various perspectives.

Metadata Estimation
Finally, the features obtained through these musical analysis processes are used to estimate metadata, examples of which are listed below. The resulting metadata can be used for song searching and other purposes.

Examples of Metadata

Perceived speed	The speed of the music as perceived by the human ear. This feature is distinguished from tempo, since the perceived speeds of songs may vary because rhythm patterns and other factors, even if the tempo is identical.
Perceived energy	The energy of music as perceived by human ears. A quiet song will seem to have less energy, while a bright and lively song will seem to be more energetic.
Genre	Whether or not the song fits a particular genre, such as rock, jazz or classical Instrumental sound: Whether or not the music includes particular instruments, such as piano, bass or guitar.
Instrumental sound	Whether or not the music includes particular instruments, such as piano, bass or guitar.
Mood	Whether or not the song fits particular mood keywords, such as "bright" or "refined."

With 12 tone analysis, a vast amount of statistical data attached to each of several dozen metadata can be subjected to statistical analysis and machine learning, resulting in extremely accurate metadata estimation.

Some musical analysis technologies can be used to distinguish between music and talk with considerable accuracy. A music/talk recognition system based on 12 tone analysis would extract basic features from the results of time-tone analyses of radio programs. After learning from actual radio broadcasts, the system would then be able to label recordings as music or talk at specific intervals.

Because 12 tone analysis classifies music using a large number of features that have been optimized for music, it is able to provide extremely accurate sorting of music that has been prone to recognition errors with earlier systems, such as rap and music with wide variation in volume.

Sony leverages Sony Group company Gracenote's music metadata distribution system as a means for Sony to attain and utilize 12 Tony Analysis metadata at speeds much faster than would otherwise be possible.

When earlier systems are used to search for similar songs on the basis of metadata distance, the resulting songs are not always similar. This is because metadata distance does not necessarily reflect similarities in music as perceived by human beings.

With 12 tone analysis, features are first converted into features that closely reflect similarity based on experimental data obtained by measuring similarity as actually perceived by human beings. Song similarity is then estimated using the converted features. When similar songs are detected in this way, the results more closely match human perceptions.

Sony will continue to develop technology to improve the accuracy of automatic metadata assignment. Another goal is the proposal of new applications based on metadata.