Renesas Electronics’ R-Car Support for cocoro SB “Emotion Recognition Engine” Enables Cars to Read Driver's Emotions and Provide the Best Response

TOKYO, July 19, 2017 ― Renesas Electronics Corporation (TSE:6723), a premier supplier of advanced semiconductor solutions, today announced that it has developed a development kit for its R-Car that takes advantage of “emotion recognition engine” (Note 1), an artificial sensibility and intelligence technology pioneered by cocoro SB Corp., a SoftBank Group company. The new development kit enables cars with the sensibility to read the driver's emotions and optimally respond to the driver’s needs based on their emotional state.

The development kit includes cocoro SB’s emotion recognition engine, which was developed leveraging its sensibility technology to recognize emotional states such as confidence or uncertainty based on the speech of the driver. The car’s response to the driver’s emotional state is displayed by a new driver-attentive user interface (UI) implemented in the Renesas R-Car system-on-chip (SoC). Since it is possible for the car to understand the driver’s words and emotional state, it can provide the appropriate response that ensures optimal driver safety. As this technology is linked to artificial intelligence (AI) based machine learning, it is possible for the car to learn from conversations with the driver, enabling it to transform into a car that is capable of providing the best response to the driver. Renesas plans to release the development kit later this year.

Renesas will also demonstrate its connected car simulator incorporating the new development kit based on cocoro SB’s emotion recognition engine at the SoftBank World 2017 event, to be held by SoftBank Corp. from July 20 to 21 at the Prince Park Tower Tokyo.

Renesas considers the driver's emotional state, facial expression and eyesight direction as key information that combines with the driver's vital signs to improve the car and driver interface, placing drivers closer to the era of self-driving cars. For example, if the car can recognize the driver is experiencing an uneasy emotional state, even if he or she has verbally accepted the switch to hands free autonomous-driving mode, it is possible for the car to ask the driver “would you prefer to continue driving and not switch to autonomous-driving mode for now?" Furthermore, understanding the driver’s emotions enables the car to control vehicle speed according to how the driver is feeling while driving at night in autonomous-driving mode. By providing carmakers and IT companies with the development kit that takes advantage of this emotion recognition engine, Renesas hopes to expand the possibilities for this service model to the development of new interfaces between cars and drivers and other mobility markets that can take advantage of emotional state information.

Based on the newly-launched Renesas autonomy™, a new advanced driving assistance systems (ADAS) and automated driving platform, Renesas enables a safe, secure, and convenient driving experience by providing next-generation solutions for connected cars.

For more information on the Car Information System, please visit: https://www.renesas.com/en/solutions/automotive/car-information-system.html

Note 1: Emotion Recognition Engine, an artificial sensibility and intelligence technology, consists of two parts. The first part is voice emotion recognition that performs an analysis of a voice signal. The second part is emotion generation technology that generates emotions for the device by forming a virtual hormone balance derived from various connected sensors. Then it makes it possible to perceive the speaker's emotion by charting an "emotion map" as both a color, for example, yellow for happiness and red for sadness, and as a volume level from 1-10.

About Renesas Electronics Corporation

Renesas Electronics Corporation (TSE: 6723) delivers trusted embedded design innovation with complete semiconductor solutions that enable billions of connected, intelligent devices to enhance the way people work and live. A global leader in microcontrollers, analog, power, and SoC products, Renesas provides comprehensive solutions for a broad range of automotive, industrial, infrastructure, and IoT applications that help shape a limitless future. Learn more at renesas.com. Follow us on LinkedIn, Facebook, Twitter, and YouTube.

(Remarks) All registered trademarks or trademarks are the property of their respective owners.

(Note) The “emotion engine” has been renamed to “emotion recognition engine” as of September 19, 2018.

The content in the press release, including, but not limited to, product prices and specifications, is based on the information as of the date indicated on the document, but may be subject to change without prior notice.