nxtAIM is a research project that encompasses all essential elements for successful AI development: comprehensive automotive data from industry and academia, powerful computing resources, technical expertise, and innovative foundation models to build upon.
nxtAIM works with Artificial Intelligence and Machine Learning methods
People and machines share the traffic space and interact with one other. Autonomous vehicles must be highly flexible and safe when on the road, understand traffic situations well and make appropriate decisions. This can only be achieved with the use of Artificial Intelligence (AI) and Machine Learning (ML) methods. The latest advancements in the development of so-called Generative AI could revolutionize automated driving.
Automated driving functions have so far been usable only within very a limited parameter space and a defined operating environment. This is due to the underlying system architectures, which process information linearly, step by step and in one direction, and the use of so-called supervised, discriminative Machine Learning (ML) methods, which require precisely annotated, i.e. labeled, data.
In contrast to discriminative methods, Generative AI can often be trained without precise annotations and can use existing annotations to guide sampling. Generative AI can therefore be used for tasks such as synthetic data generation to fill data gaps and for simulation.
As an outstanding result for industrial application, foundation models for driving data will emerge, which have not existed in this form before and will significantly boost the further development of autonomous driving.
Successful AI development and innovations in autonomous driving
Sampling – 3 success factors:
methods, data, consortium
“Generative methods allow estimating highly complex probability distributions from given data points and drawing new scenarios from them. In recent years, generative methods have reached a level of maturity where synthetically generated image data can no longer be visually distinguished from actual photos. Further, it is possible to control the sampling process by a variety of influencing parameters.
The goal of nxtAIM is to adapt these methods for use in autonomous driving. In addition to the above mentioned technological maturity of the methods, the amount of data available in the consortium and the expertise of the scientific and industrial partners are important success factors for this project.”
Dr. Jörg Reichardt
nxtAIM project coordinator, Continental
The nxtAIM story
The nxtAIM story
Challenge 1
Autonomous Driving Functions are of limited use
Today, engineers and scientists face the challenge that fully automated driving functions are only usable in a limited scope, for example, on highways in good weather, for parking in parking garages, and at low speeds on predefined routes. The reason lies in how the data for automated driving is processed systemically: the flow of information is linear. Data is processed step by step in one direction—from perception and interpretation to planning and execution.
Innovation 1
Bidirectional information flow in the chain of effects
The project team is taking a novel approach with an innovative idea: Generative AI methods are being used to enable bidirectional information flow. This means that information can not only be passed on but also fed back and immediately verified. This increases the scalability and transferability of autonomous driving functions and improves the traceability of processes.
Challenge 2
Increasing the cognitive capabilities of autonomous vehicles
One of the greatest challenges is how to enhance the cognitive capabilities of autonomous vehicles so that they can operate safely and reliably on the road at all times. Discriminative AI methods require large amounts of precisely annotated training and test data, which can only be economically generated within a limited operational environment. Generative AI, which has already proven itself in large language models and text-to-image generators, opens up new possibilities. Researchers aim to use this technology to fundamentally change and advance the system architecture of automated driving.
Innovation 2
Development of foundation models for autonomous driving
Foundation models are versatile. They help to process data more efficiently, intelligently and transparently and to increase the scope for action. nxtAIM will train these models with relevant vehicle data collected from various locations (sensors) in the vehicle, significantly accelerating the development of autonomous driving functions.
Outlook
A paradigm shift in autonomous driving
Ultimately, the nxtAIM project will lead to a paradigm shift in the development of automated driving. The vision to operate autonomous vehicles safely and reliably in the open world without any restrictions is approaching reality.
Facts & Figures
Project Budget
43,5 Mio. €
Consortium Lead
Dr. Jörg Reichardt
Continental
Dr. Ulrich Kreßel
Mercedes-Benz
Consortium
20 Partners
OEMs, suppliers, technology providers, research bodies, external partners
Funding
27 Mio. €
Duration
36 Months
January 2024 – December 2026
nxtAIM as part of the KI Familie of the VDA Leitinitiative
The project families, KI Familie, Pegasus, and @City, developed by the Lead Initiative, address fields of research and development that are crucial for the competitiveness of autonomous and connected driving. While the KI Familie focuses on data generation and safe AI, the project work in the Pegasus family revolves around standards for the testing and safety validation of highly automated driving functions. In the @City family, the focus is on the expansion of automation and operational design domains (ODD) with a focus on urban traffic. Current developments in AI are leading to a paradigm shift in the application of autonomous driving functions. With a new generation of projects within the KI Familie, the VDA Leitinitiative seizes the opportunity to once again take a leadership role in autonomous driving through broad cooperation among partners from industry and academia, leveraging generative AI technologies.
