Search

On the road to autonomy, the crucial role of FPGA – Interview with Xilinx

At home, I speak to my virtual assistant to get the weather, to play some music, to answer a call. I unlock my smartphone by just looking at the front camera and I am talking to it to get direction. Thanks to Artificial Intelligence, particularly Neural Networks-based algorithms, life is easier, at least for me. What about AI for automotive? Will the promise of reading my emails while my car drives me to work will be held? Will my taxi drive me to the restaurant without any driver? Yes, but as in consumer applications, it needs specific computing hardware.

In the report Artificial Intelligence Computing for Automotive 2019, Yole Développement (Yole) describes the autonomous car market as divided into two segments: cars as we know, gradually integrating more features making them more and more autonomous and robotic cars, already fully autonomous at low speed and in designated areas, agnostic in terms of sensor and also of computing hardware, choosing the best of what it exists. Over the next 10 years, with the development of robot taxis and shuttles, this market will remain the main revenue generator for AI in automotive, with $9B in total computing revenue expected in 2028. In 2019, the first cars qualified as “ADAS level 3” will hit the road, and AI will enhance ADAS level 2+ cars, replacing conventional computer-vision algorithms. Yole expects a $63M computing market for ADAS in 2019, reaching almost $3.7B in 2028.

One of this specific computing hardware is Field Programmable Gate Arrays (FPGA). In order to offer our readers a better understanding in this technology, Yohann Tschudi, PhD, Software & Computing Market & Technology analyst at Yole, interviewed Willard Tu, Senior Director at Xilinx. He explains more about the role of this particular type of hardware in the autonomous future.

Yohann Tschudi, PhD (YT): Could you please introduce yourself and your division to our readers?

Willard Tu (WT): I am responsible for the global automotive business at Xilinx.

I have been in the automotive market for more than 25 years, and before Xilinx I worked at Arm (now Softbank), NEC (now Renesas), National Semi (now TI) and Motorola (now NXP). As you can see the semiconductor industry is a shrinking, and the automotive circle is even smaller.

YT: FPGA had the reputation to be a tough hardware, with difficulties to find developers, and used in education or in very specific cases, how and why do you think this image is changing?

WT: Volume, innovation, and rapid change.

From a volume perspective semiconductor companies want high volume to fill the capacity of their fabs. But innovative features like ADAS or autonomous drive do not have the same volume profile as powertrain, body, airbag ECUs, etc. Hence it is difficult for the market to create a highly integrated and cost optimized solutions. To further complicate things, there are fewer and fewer semiconductor companies and even fewer that own their own fabs. There has to be more focus on innovation and differentiation. So when volumes are still growing, the FPGA is a perfect fit.

Innovation and rapid change both benefit from FPGA adaptability. When you are in the innovation stage there is no standard way to make something. This is where we are with ADAS and autonomous drive. Airbag ECUs are essentially the same capability and content from one supplier to another but you cannot say that for the emerging technologies.

FGPAs can enable the innovation, adapt to the rapid changing in requirements or AI techniques, and be cost effective while providing those benefits.

YT: Can you explain us how Xilinx chose to enter in the automotive market in general and ADAS in particular?

WT: Xilinx entered the automotive market more than 14 years ago. More than 160 million Xilinx devices are in automotive systems today, and approximately 55 million of these are used for ADAS alone.

ADAS and autonomous drive require a greater need for the ability to perceive what’s happening around the vehicle in all driving conditions. Xilinx is the ideal solution to handle these needs through our flexible and re-programmable automotive platforms.

YT: Can you present your hardware solution and software stack?

WT: The beauty of FPGA technology is that it is a blank sheet for an engineer to create their own hardware and make a highly differentiated solution. The same SoC like our popular Zynq MPSoC ZU3 is being used in forward looking camera, surround view and driver monitoring systems today. It is also being designed into LiDAR and 4D imaging radar. Our SoC is a “chameleon” that adapts to many different applications needs.

Xilinx chose to license processor architecture from ARM which is well-aligned with our automotive customer base. Practically all of the primary automotive OS providers have ported their products to Xilinx devices, including eSOL, Mentor, Greenhills, Vector, Micrium, QNX, Sysgo, Windriver, and others. It is important to note that this covers standard automotive operating systems and hypervisors. For AUTOSAR, Xilinx provides a Microcontroller Abstraction Layer which enable AUTOSAR OS providers to port their products to Xilinx devices.

Xilinx also provides firmware for elements within our devices such as the Platform Management Unit.

YT: For you, what are the pro and cons of using FPGA in such a domain?

WT: The pros of using Xilinx technology in applications such as ADAS and autonomous drive can vary depending upon specific goals of the customer. Speaking from a strict silicon performance perspective, the ability to create parallelized, simultaneous processing pipelines can lead to lower latency and lower power implementations. Lower latency is obviously critical in ADAS and autonomous drive.

Low power means less thermal loading which can be an issue for modules mounted in the windshield, headliner, and other high ambient areas. By parallelizing computations, clocks can run slower thereby reducing dynamic power consumption for a give through-put.

Speaking of through-put, you can speak about theoretical operation-per-second of a device, but the real value to customers is how much your computational engines are utilized – i.e. the difference between theoretical and practical performance. Xilinx programmable logic contains a tremendous amount of routing links as well as a large number of small memories. The combination of these resources allows designers to create customized data feed networks to their computational engines to produce high levels of utilization.

The advantages of Xilinx automotive devices goes well beyond the performance parameters we have discussed. First, Xilinx programmable logic provides customers the flexibility to adapt to the evolving needs in an emerging application space like ADAS and autonomous drive. Taking advantage of improved interface standards, algorithm innovation, and new sensor technologies all require an adaptable platform that can support not only SW changes, but HW changes as well… and that’s exactly what Xilinx devices provide.

Second, is scalability. We always qualify a scalable group of devices from a particular Xilinx product family. These devices vary the amount of programmable logic and many times come in pin-for-pin compatible packages. This means a developer can create a single ECU platform to host low, mid, and high versions of ADAS feature bundles and scale the cost as needed by choosing the minimum density device required.

Finally, and for some customers most importantly, Xilinx devices allow developers to create unique, differentiated processing solutions that can be optimized to a particular application or sensor. This is simply not possible with ASSP devices – even those that offer dedicated accelerators are limited in how they can be used and are basically offered to all of one’s competitors. Xilinx long-time customers have created high-value IP libraries that only they have access to and those functions can be used by various products across the company.

As for the cons, much of them come from historical perceptions on cost. Many think of FPGA technology being fine for prototyping, but not for production. Beginning with the 90nm node, Xilinx devices have become very cost effective for automotive volumes – we would not have sold over 160M units into the industry if they were not.

YT: We are seeing AI everywhere nowadays and it is rapidly entering the consumers’ daily life. In your opinion, why and how will AI change the game in the automotive market?

WT: AI – more specifically neural nets (NN) will provide a lower cost to develop, and lower hardware performance requirements which means lower cost. I have seen many traditional CV approaches that require more performance than NN such as forward looking camera and driver monitoring systems. If we are to achieve great adoption I believe NN will play a bigger role.

YT: Yole Développement’s analysts distinguished two different automotive markets: ADAS vehicles, with an adoption of fusion sensor computing platforms, and robotic vehicles, using already complete computer systems equivalent to datacenter platforms.
In your opinion, what trend will be most followed by the OEMs in the ADAS field which is highly conditioned by regulations and low margins?
Are you in a position to gain a share of this market, considering it could be a high-volume market with low prices?

WT: Price is always an important dimension when you are dealing with automotive volumes. Right now, the focus is on innovation, but in time (and within the next decade) cost will become a bigger factor.

ADAS needs scalability – one size does not fit all. Surround view is a great example. You can have bird’s eye view, bowl view, object detection, object classification, automated park assist, tail gate monitoring, and valet parking as features. As you add features you will either need more a complex SoC or more scalable FPGA.

YT: If we look at the other segment, how Xilinx is positioned for robotic vehicles?

WT: There is so much focus on the AI which we play well in, as well as Data Aggregation, Pre-processing and Distribution (DAPD). The DAPD functionality interfaces with the different sensor modalities to perform basic processing, routing and switching of information between processing units and accelerators within the processing unit.

We have some customers using up to 10x devices to do this, and some just using 1x.

Courtesy of: Xilinx

Our recent announcement with ZF highlights the ability of our technology to go beyond the DAPD function to now be utilized for AI compute.

YT: Today, some robot taxis and shuttles are in use, though it is still in experimentation stage and is not yet a common way to travel.
How quickly do you expect robotic vehicles to become wide-spread on our roads? What are the obstacles to this expansion? Regulation? Accidents?

WT: There are a lot of obstacles…but let me answer by asking you a question. Have you experienced a robo-taxi? I can tell you from riding in a number of our customers’ vehicles that the technology is there, it just needs either a little “assistance” or a little more “time”.

  • Assistance meaning a sand-boxed operating area where it can have an embedded localized HD Map or data from infrastructure.
  • Time meaning, we need to get to lower cost points which need to see more volume, sort of the chicken and egg problem.

I do believe that governments will limit the adoption to help to minimize the societal impact and disruption that autonomy could bring. But that is a problem when we get to a mass deployment phase. We are not at the point yet.

YT: To end off, let’s talk a bit about the future. What Xilinx imagine in terms of number of sensors and cameras? Will FPGA will be able to treat the huge data flow coming from these sensors?

WT: This is a big advantage for Xilinx. As I mentioned before our high value role is with DAPD. This allows us to adapt to the number of sensors, total count of different types. It allows us to adapt to the capability of the sensors. It allows us to adapt to the different interfaces for each sensor type. That adaptability is dynamic, we can change mid-stream in the product life cycle, which is great for robo-taxis which could get a complete hardware retrofit during its life cycle.

YT: What move do you think Xilinx should do to stay in the business and competitive particularly in front of ASICs or sensor fusion platforms?

Courtesy of: Xilinx

WT: It is more expensive to create silicon ASICs but despite this we have a secret weapon. Dynamic Function eXchange. This is unique to FPGA technology. It allows us to re-program a function as the device is being utilized swapping mutually exclusive functions. For example an FPGA can be a front camera design when in highway drive mode, but that same FGPA can be re-programmed to be automated parking/surround view when in low-speed mode. This means for one FPGA you can replace two ASICs.

This is just the beginning…imagine what we can do for in-cabin monitoring.

Courtesy of: Xilinx

YT: What are your thoughts on the vehicle of 2030-2035?

WT: By 2030/2035, autonomous fleets will be pervasive as first adopters. On the personal-owned vehicle, I think we may see some autonomous cars on the ultra high-end, but not in mass production.

YT: Do you think our kids will still need their driver’s license?

WT: Hmmm…I think we’ll have to wait and see on this. However, the new generation of kids are definitely more concerned about their smartphones than they are about getting their driver’s license.

YT: Is there anything else that Xilinx would like to add?

WT: If you are not already working with us on ADAS or autonomous drive, give us a call. Stop re-inventing the wheel – redesigning over and over moving from one ASIC SOC to another ASIC SOC. Stop worrying about redesigning IP from the edge to the central domain controller. Stop trying to use different SoC platforms for each different sensor.

FPGA technology will help you save your most valuable resources – your engineering talent. We will can give you economies of scale as one device family can support front camera, LiDAR, 4D radar, surround view and driver monitoring. FPGAs can do it all.


Interviewee:

Willard Tu is a Senior Director at Xilinx, where he leads global business development, product planning, and marketing strategies for the company’s automotive business. Tu has spent over two decades at the axis of the semiconductor, automotive and computing industries. He was previously at Arm, where he evangelized CPU IP and developed ecosystems to support Arm’s growth in automotive. At NEC Electronics, (now Renesas), Tu led the North American Automotive sales and marketing teams, growing sales to over $150 million. Tu holds a BS degree in Electrical Engineering from the University of Michigan and an MBA from the University of Phoenix.

Interviewer:

As a Technology & Market Analyst, Yohann Tschudi, PhD is a member of the Semiconductor & Software division at Yole Développement (Yole). Yohann is daily working with Yole’s analysts to identify, understand and analyze the role of the software parts within any semiconductor products, from the machine code to the highest level of algorithms. Market segments especially analyzed by Yohann include big data analysis algorithms, deep/machine learning, genetic algorithms, all coming from Artificial Intelligence (IA) technologies.
After his thesis at CERN (Geneva, Switzerland) in particle physics, Yohann developed a dedicated software for fluid mechanics and thermodynamics applications. Afterwards, he served during 2 years at the University of Miami (FL, United-States) as a research scientist in the radiation oncology department. He was involved in cancer auto-detection and characterization projects using AI methods based on images from Magnetic Resonance Imaging (MRI). During his research career, Yohann has authored and co-authored more than 10 relevant papers.
Yohann has a PhD in High Energy Physics and a master degree in Physical Sciences from Claude Bernard University (Lyon, France).


Related report:

Artificial Intelligence Computing for Automotive report, Yole Développement, 2019

Artificial Intelligence Computing for Automotive 2019
Artificial Intelligence for automotive: why you should care




Related webcast:

AI’s Impact on the Automotive Industry: Trends, Market, Players, and Future
Artificial intelligence shortens the path to autonomy and brings the home into the car.

up