https://www.epo.org/en/news-events/press-centre/press-release/2019/451963

Computer vision for greater road safety: Amnon Shashua and team named finalists of the European Inventor Award 2019

07.05.2019

in Press release

Israeli inventor Amnon Shashua and the Mobileye team nominated for European Patent Office (EPO) prize for their Advanced Driver Assistance System
Invention uses a single camera and artificial intelligence to alert drivers to potential hazards
Mobileye system is already used in more than 40 million cars worldwide, bringing lifesaving driver assistance technology to a wider market than ever before

Munich, 7 May 2019 - The European Patent Office (EPO) announces that Israeli computer vision expert Amnon Shashua together with his team - Erez Dagan, Yoram Gedalyahu, Gaby Hayon, Elchanan Rushinek, Shai Shalev-Shwartz and Gideon Stein - have been nominated for the European Inventor Award 2019 for developing an automotive safety system that uses a single camera and image recognition software to enable cars to see, interpret and react to dangers on the road.

The co-founder and Chief Executive Officer of Mobileye and his colleagues have been named as finalists in the category "Non-EPO countries" for their invention, the first-ever driver-assistance system on the road to be able to identify vehicles, pedestrians, cyclists, lane markings and traffic signs. The system the team pioneered is simpler and more economical than rival systems, enabling it to be used in more cars and bringing safety benefits to many more people. Commercialised by Mobileye, the company Shashua co-founded in Israel in 1999 and which now operates as a subsidiary of Intel, the technology has already been installed in more than 40 million vehicles around the world - from economy to luxury models - produced by most of the major car manufacturers.

"Professor Shashua and his colleagues developed a simple yet smart solution for advanced driver assistance systems using artificial intelligence that pays high social dividends," said EPO President António Campinos announcing the European Inventor Award 2019 finalists. "This invention has not only had a significant impact on the automotive market, but is also saving countless lives."

The winners of the 2019 edition of the EPO's annual innovation prize will be announced at a ceremony in Vienna on 20 June.

Inspired by human vision

According to World Health Organization statistics, more than a million people die every year in road accidents. The most common causes of collisions include drifting out of lanes, rear-end collisions and speeding - all hazards that electronic measures known as advanced driver-assistance systems (ADAS) can nowadays detect and prevent. In recent decades, ADAS have built on progress in sensors, computing power and artificial intelligence to autonomously monitor, warn, brake, and steer cars out of harm's way. But ADAS had previously used hardware such as twin camera systems calibrated to collect stereoscopic images, or emitters of radar-like pulses that map out surrounding objects. These solutions were expensive, complicated and cumbersome.

The system developed by Shashua and his colleagues is different in that it is inspired by human visual perception, something the Israeli engineer had long taken an interest in, having completed a PhD in brain and cognitive sciences at the Massachusetts Institute of Technology. Since 1996 he has been a faculty member at the Hebrew University of Jerusalem, where he currently serves as Sachs Professor of Computer Science. He has published more than 120 academic papers in the fields of machine learning and computer vision.

Shashua explains computer vision as "teaching machines to interpret their environment as a person does with their eyes". Accordingly, his founding philosophy for Mobileye was that if humans can drive a car based on vision alone, so could a computer.

Realising that humans with vision in only one eye are fully capable of accurately perceiving driveable paths, Shashua and his team were determined to prove that a single lens camera was capable of serving as the primary sensor to enable ADAS and future autonomous driving: "Just like our eyes allow us to see the world and understand our environment using only vision, we believed that by applying sensing techniques to a monocular camera, a computer could do the same. The challenge we faced was teaching a computer to see and understand a variety of road users, markings and environments in order to properly compensate for human error."

A smarter solution for automotive's greatest challenges

One of Mobileye's first clients, a major global automaker, asked for a system that could detect lanes to avoid collisions. In stark contrast to other ADAS providers, Shashua and his team were intent on solving the problem using a single camera: "We knew vision-based ADAS could revolutionise road safety. We were determined to perfect the system with a single camera because we saw the transformative potential of this highly differentiated approach. In fact, the human visual system relies on a diverse set of cues, which are monocular in nature, like perspective, motion, shading and context. Although it sounds unintuitive, doing this with a single camera, while employing processes that mimic the human brain, would not only make the system less expensive but also work better than a stereo camera relying mostly on geometric triangulation," Shashua says.

In the system Shashua and his team developed, a single-lensed front-facing camera integrated into the car's windshield scans the road ahead and streams footage to a tailor-made computer processor on board the vehicle where artificial intelligence software detects road features. Sophisticated algorithms that are built into the hardware compare changes in the relative size and position of nearby objects every 27 miliseconds to calculate their speed and trajectory. The technology uses this data to predict driving hazards in real time and send warnings to the driver or safety commands directly to the vehicle to automatically react.

Within the first year of founding, the Mobileye workforce had patented inventions on how to compute collision times and reduce measurement noise. As the company teamed up with more automakers, it faced greater challenges to solve using computer vision.

Detecting lanes was only the beginning. In response to customer demand for more features that would further enhance the safety of drivers the team advanced the detection capabilities of its custom system-on-a-chip, the EyeQ^®. Intent on pioneering new safety features, Mobileye also patented solutions for detecting road users such as bikers and pedestrians, as well as traffic signs.

Thanks to improvements in the software, higher computational processing power and more powerful cameras, the Mobileye team continued to develop the technology, for example by extending the distance at which the system could detect vehicles and pedestrians. The engineers also developed free-space detection, which used semantic cues beyond simple lane markings to provide detailed information on the space that is available to drive.

These advancements have also paved the way for the company to establish a leading position in the development of fully autonomous vehicle technology. At the EPO, this sector saw a 330% increase in the number of patent applications filed in the past seven years, and the market is expected to be worth up to EUR 62 billion by 2030. More than a dozen automakers have chosen to collaborate with Intel and/or Mobileye for their autonomous vehicle programmes.

Mobileye has filed approximately 150 patents so far internationally. These have proved important in helping the firm secure clients across the automotive industry, as car companies have reputations to maintain and are keen to avoid intellectual property disputes.

Towards a zero accident future

By shifting the core of ADAS from heavy hardware to smart software, the Israeli inventors managed to reduce the cost of their technology and boost its efficiency. The cost savings meant that the technology could be extended from its original market, luxury cars, to mid-range and economy vehicles, bringing its safety benefits to millions of drivers worldwide.

Mobileye held some 70% of the global market for obstacle detection software in 2017. The company works with approximately 25 leading auto manufacturers, including BMW, Volkswagen Group, Ford, and General Motors, powering safety features such as automatic emergency braking and lane-keeping assistance.

In 2017 Mobileye's innovation was recognised on a global scale when Intel acquired the firm for EUR 13.5 billion, the largest ever acquisition of an Israeli company. The company has since doubled its workforce and now employs more than 1 000 people. It debuted in the public markets in 2014 when it listed on the New York Stock Exchange, to date the biggest IPO of an Israeli company in the US. Currently Shashua serves as Senior Vice President at Intel Corporation, and President and Chief Executive Officer of Mobileye. The Mobileye team nominated for the European Inventor Award 2019 also includes: Erez Dagan, Executive Vice President Products and Strategy at Mobileye and Vice President Intel Corporation; Yoram Gedalyahu, Vice President mapping and localization and co-General Manager REM; Gaby Hayon, Executive Vice President R&D at Mobileye and Vice President Intel Corporation; Elchanan Rushinek, Executive Vice President Engineering at Mobileye and Vice President Intel Corporation; Shai Shalev-Shwartz, Chief Technology Officer; and Gideon Stein, Chief Research Scientist.

The global market for ADAS is vast and growing rapidly. In addition to the booming passenger vehicle market, Mobileye expects potentially rapid adoption of ADAS in commercial vehicles due to changes in legislation and higher safety standards among fleet vehicle operators. In addition to the road safety benefits, the technology can also lead to shortened journey times, bringing fuel savings for car owners and contributing to CO₂mitigation.

Preventing road accidents, though, remains a key motivation for the Mobileye team in their work. Driver assistance systems have the potential to reduce the number of traffic accidents considerably - up to 46% according to an EU study published in 2016. Shashua and his colleagues are committed to developing the technology further with the aim of eventually bringing road casualties down to zero in the future:

"Technology advancements of the last decade have demonstrated that most road collisions can be eliminated. We are proud that our developments are contributing to a new reality for automotive safety. We have no plans to slow down until all vehicles are properly equipped with lifesaving ADAS technology."

Additional resources

View the patent(s): EP1806595, EP1236126, EP1741079

Additional information, photos and videos about the European Inventor Award 2019 can be found in the EPO Media Centre. Smart TV users can download our app "Innovation TV" and watch videos about all finalists on their TV screen. The award ceremony on 20 June 2019 will be broadcast live on Innovation TV, the EPO website and the EPO's Facebook page.