A broad definition of face recognition encompasses a variety of technologies used to extract data about facial images. Sony's face recognition technology offers the following features.

Face detection:	The detection and indication of facial zones that are facing in various directions in complex scenes.
Facial pose estimation:	The estimation of the direction (angle) to which a face is turned
Facial part detection:	The identification of the positions of facial parts such as the centers of the eyes, the tip of the nose, and the corners of the mouth
Facial attribute classification:	The classification of faces by gender, ethnicity, age, expression and other characteristics
Face identification:	The identification of individuals through comparisons with registered people (This is the narrow definition of face recognition.)
Multi-view face detection:	In this example, even faces that are not turned toward the front have been detected, and facial poses have been estimated at the same time. The direction of the arrows indicates the direction of the face, and the length of the arrows indicates the value of angle. (An angle of zero denotes a frontal pose, while a larger angle indicates that the face is turned to one side.)

The following description relates to the type of face recognition that is most commonly used in commercial applications.

The first step is to define a facial pattern of a specific size. Human vision can judge whether or not a face is present even in a low-resolution image made up of 16x16 pixels. This ability does not rely on color, and human eyes will find faces even in a monochrome image. Computers process facial patterns using images of about the same size.

1. Detection of face to be scanned
The system scans the image from top left to bottom right until it finds this pattern.

2. Facial pattern classification
Facial patterns are not easy to define. They vary from person to person, and they also change according to the angle of the face and differences in lighting conditions or facial expressions. To overcome this, it is necessary to formulate functions that allow discrimination between facial and non-facial images by applying statistical methods to large numbers of facial and non-facial images. The key to this is the use of features. To facilitate the implementation of this technology on consumer electronic products, Sony uses a unique set of extremely simple features. Because these features can be combined in various ways, it is possible to achieve powerful pattern classification performance despite the simplicity of the operations involved.

Analysis takes longer. However, by using a PC cluster system installed for use in artificial intelligence research, Sony has been able to detect optimal features and compile a compact dictionary of facial patterns.

Sony is already using face recognition technology in a variety of AV equipment in the following categories.

Better Photographs:
The Face Detection function on Sony's Cyber-shot digital still cameras rapidly locates faces in each shot and uses camera controls, including AF, AE and AWB, to optimize the settings for superb reproduction of facial areas. The Smile Shutter function adds the ability to scan facial attributes, detect smiles and automatically control the shutter timing.
The Auto Touch-up function on Sony's PictureStation photo printers detects facial areas before printing and applies a range of imaging processes, including backlight correction, skin color adjustment and focus correction.

Search:
The FaceIndex function on Sony's Handycam digital video camera detects faces during shooting and produces thumbnail images. The user can start playback from the point at which a particular face appears simply by selecting that face on the touch panel.
Sony's Cyber-shot DSC-G1 works with bundled software to detect the presence of facial images and determine their size and other parameters, allowing the user to search for files containing pictures of faces. VAIO Movie Story provides a similar capability.

Display:
The slide show function (x-PictStoryHD) on Blu-ray disc recorders and Sony's Sugoroku DVD-HDD recorder uses face detection results to support the creation of powerful face framing effects. Similar capabilities are available through the portrait slideshow function on PLAYSTATION 3, VAIO Movie Story, and on HDD Photostorage.

Currently the most advanced applications of this technology have been implemented on digital still cameras made by various manufacturers. The technology is also used extensively in the digital imaging field, including printers. Sony has led the way in taking the technology to the next stage with its Smile Shutter function, which not only detects faces but also discriminates between facial expressions. Sony will continue to expand the range of face recognition information supplied by its various products. The resulting capabilities, including editing, content searching and content production, are expected to form the basis for important new applications of this technology in the future.

Face recognition technology is likely to be implemented on a variety of equipment in the future. Sony's priority will be to enhance user convenience by developing new ways to manage metadata describing facial characteristics in faces detected in images, and to share that metadata among devices.