If you are inside a new car there is a big chance you are being filmed. This statement might scare us at first, but the reality is not an Orwellian surveillance of your private space but rather a real-time machine analysis of your attention as a driver or your behavior as a passenger.
Depending on where you are, such technology is either already mandatory in any vehicle coming out from production or will be soon. This is motivated by the clear trade-off between saving lives and, as we would see below, a simple and common technology. For example, driver impairment or distraction contributed to almost 27% of 2018’s fatal accidents in the UK (source: https://www.data.gov.uk/dataset/cb7ae6f0-4be6-4935-9277-47e5ce24a11f/road-safety-data) and contributed to about 3,000 deaths every year in the US between 2010 and 2019 (source: Distracted Driving | Transportation Safety | Injury Center | CDC).
In addition, autonomous vehicles with hands-off or eyes-off have begun to enter the market. These automated driving systems need to be handed off to the driver at the end of the operation. The driving monitoring system is also necessary for such a system to recognize the status of the driver.
Technology Trend
Camera sensors are the only technology that can detect all types of impairments (i.e., using the phone, eyes closed, etc). This makes them integral to any interior monitoring solution but can be complemented with either radar or multi-camera sensors to cover obstructed areas like rear-facing child seats and behind the front seats.
In addition, any monitoring needs to work in all light conditions since logically we tend to fall asleep at night when visibility is low (and a lit cockpit reduces driver’s visibility and is very uncomfortable for passengers). Thus, infrared sensors become the ideal solution.
Broad Feature Set
Customers look at their cars differently and their expectations vary a lot: some would be looking for a cost-effective vehicle with the basic and minimum equipment, others want the safest ride for their families, while others expect the latest technology and features for their driving experience.
Part of the E/E Evolution
Interior monitoring introduction comes at an exciting time (a.k.a. challenging):
- Vehicles are no longer a hardware-driven uniform product coming out from the production line, but a connected personalized device with new functions and continuous upgrades
- We are moving from a distributed architecture with local processing toward a centralized processing
Where does interior monitoring fit in all that? Is it a cost-effective stand-alone system equipped with a mandated or feature-rich system where users subscribe or purchase features like video calls and seat adjustments based on morphology?
Renesas’ purpose is to make our lives easier, we thus offer a broad range of solutions for interior monitoring to try and meet all these needs while addressing all these different factors.
- R-Car SoCs scale up from the low-cost standalone driver monitoring for the mandatory features, to include end-user features and up to cross-application integration with advanced driver-assistance systems (ADAS), in-vehicle infotainment (IVI), and vehicle computers.
- You can then choose to best fit solution whether you are looking for a standalone vs. centralized solution as well as a basic implementation vs. state-of-the-art video processing with AI
State-of-the-Art Integration and Energy Efficiency Challenge
Integration inside the rear-view mirror became a state-of-the-art solution for standalone driver monitoring, being an elegant and non-intrusive location for the field of view covering most of the cockpit. This brings heat dissipation constraints that add to the constantly challenging energy efficiency in tomorrow’s vehicles.
Highly Efficient Standalone Driver Monitoring System (DMS)/Occupant Monitoring System (OMS)
Thanks to its long track record in automotive smart cameras and best-in-class TOPS/W AI capabilities, R-Car offers highly efficient interior monitoring (i.e., 2W for mandatory features and below 4W for premium 5MPix RGB-IR DMS+OMS by leveraging its rich set of hardware accelerators.
Implementation Example
