Kao develops original skin evaluation AI "Kirei skin AI" that also learned human sensibilities

 Kao Makeup Research Institute has further learned human sensibilities through "Skin Evaluation AI," which evaluates and visualizes diverse and delicate skin textures using deep learning, which is one of the artificial intelligence (AI) technologies. We have developed "Kirei Skin AI" that combines detailed skin analysis with human perspectives and judgments. Some of the results of this research were presented at the "2021 Summer Games of the Visual Society of Japan" (held online from September 22 to 24, 2021).

In January 2021, Kao evaluated and visualized diverse and delicate skin textures as the first step toward human sensibilities to perceive changes in impression by reading various information from slight differences in skin color and texture. It is reported that it has developed "Skin Evaluation AI".

This technology is characterized by learning the small area skin image "skin patch" cut out from the face image by deep learning, which is one of the AI ​​technologies, "bare skin and makeup skin" and "immediately after makeup". It is possible to identify slight differences in texture in various skin conditions, such as "after a while". However, on the other hand, in order to accurately evaluate the impression of the skin when making up, it was necessary to learn from the human perspective, "How do humans feel when they see the skin?"

For example, one of the skin textures from which the human point of view is very important is "gloss", which expresses the luster of the surface, but the unfavorable luster on the skin is called "shiny", which is distinguished from the beautiful luster "gloss". Will be done. When comparing matte skin, shiny skin with makeup breakage, and smooth and beautiful glossy skin, the AI ​​developed in the past could only give shiny skin a "strong luster" rating. It can be judged as "not glossy" because it has the characteristics of the human eye that can accurately capture the impression of the skin.

Kao has a mechanism for "expert judges" who have been trained in visual evaluation to evaluate skin color, luster, skin troubles, etc., and can accurately evaluate skin. Therefore, this time, by letting the existing "skin evaluation AI" learn the "judgment of a professional judge" which is the human sensibility itself, we tried to develop an AI that has both a precise analysis of the skin and a human viewpoint / judgment. ..

Aiming to build an AI that can predict the visual evaluation of a professional judge, 10 items related to skin color and texture such as "gloss", "transparency", and "smoothness of the skin surface" among the items that can be evaluated by a professional judge will be AI. I learned.

For learning, 9306 skin patch images cut out from the facial images of 83 Japanese women aged 20 to 39 and the visual evaluation results of a professional judge for those images were used. As a result, we succeeded in constructing an AI that can predict 10 visual evaluation items of a professional judge with an accuracy of around 0.7 correlation coefficient.

Through this study, 10 visual evaluations that reflect human sensibilities were added to the existing "skin evaluation AI", and "Kirei skin AI" was born, which enables a total of 16 skin impression evaluations.

The newly developed "Kirei Skin AI" can predict the glossiness of the skin, which was difficult to objectively evaluate with a device, with high accuracy, in accordance with the criteria of gloss evaluation (visual gloss) of a professional judge. Became.

Furthermore, in order to capture the multifacetedness of gloss, the visual gloss index and other indicators related to gloss and gloss evaluated by the existing skin evaluation AI are output at the same time. Using 1596 facial images of 266 Japanese women before and after applying make-up, AI was used to evaluate three evaluation items that are thought to be related to visual luster and luster, and the relationship between the evaluation items was confirmed.