News

AI4CCAM interviewed Karla Quintero, SystemX, leader of WP1 of the project focusing on “AI-based integrated Trustworthy AI framework for CCAM”.

Karla is Senior Research Engineer/Systems Engineering Architect at SystemX.

In this interview, Karla tells us more about the most innovative aspects of AI4CCAM and the regulation of the use of Artificial Intelligence

As the project manager of AI4CCAM for IRT SystemX, what do you foresee as the greatest innovation of the project?

The AI4CCAM project addresses the assessment of trustworthiness in artificial intelligence for Cooperative, Connected and Automated Mobility. A very strong element for innovation is a methodology including the application of the scenario approach and allowing to evaluate key requirements proposed at a European level for trustworthy AI. The results in the project should provide richer knowledge on the challenges and means for evaluating these key requirements given that, to this day, no consensus is yet established and it is then a very active field of research.

The use cases in the project provide solid ground to apply this methodology as they are complementary approaches in the mobility scope, namely: AI for trajectory prediction and, on the other hand, user acceptance from the perspective of Vulnerable Road Users.

In Europe, the trend is to regulate the usage of AI. How do you see that coming in the Automotive Industry?

In the automotive industry, a layer of regulation of such a technology is already ensured intrinsically though traditional safety evaluation procedures.

However, many actions are already under way such as initiatives related to the explainability of AI algorithms embedded in vehicles. Some of the undergoing work, also tackled by IRT SystemX (among others in the PRISSMA project – Research and Investment Platform for the Safety and Security of Autonomous Mobility), consists in providing recommendations to regulate certification of AI-based systems, going from individual sensors up to the entire vehicle, and fleets of vehicles. In this sense, regulation emerges from surpassing the black box paradigm of AI, and requiring explainability all the while protecting intellectual property from providers. This is both the goal and the challenge, which is why intellectual property traceability has become a topic of high interest in the automotive industry.

Another issue in the automotive industry, as in many others, is the protection of privacy which has been of interest before and even more after the GDPR (General Data Protection Regulation) came into effect. In this sense, AI is consequently, yet partly, regulated. In this regulation scope, similar initiatives are under research regarding cybersecurity.

Some initiatives that can be mentioned include the very well-known AI-Act but also the “Grand Défi”, which “[Ensures] the security, reliability and certification of systems based on artificial intelligence”. It aims at building the process, methods and tools to guarantee the trustworthiness placed in products and services that integrate artificial intelligence. Consequently, it aims to provide a “technical” framework for the future European regulation proposal on AI. As part of this “Grand défi”, Confiance.ai brings together a group of 13 major French industrial and academic partners (Air Liquide, Airbus, Atos, Naval Group, Renault, Safran, Sopra Steria, Thales and Valeo, as well as the CEA, Inria, IRT Saint Exupéry and IRT SystemX) to take up the challenge of industrializing artificial intelligence for critical products and systems.

What is coming in the next few months?

Future advances expected in the ecosystem are namely:

• Application of AI-based systems for User Experience assessment, e.g. passenger monitoring, situational awareness, alerting passengers,
• Addressing trustworthiness through normalized levels and scales for AI-based systems,
• Using the scope of CCAM in order to extend the ODD, increase reliability of current ODD, and normalize interaction with law enforcement authorities.

Some upcoming events of interest where IRT SystemX contributes in the European scope are:

• “Announcing the European AI, Data, Robotics Forum” https://adr-association.eu/blog/announcing-the-european-ai-data-robotics-forum-08-09-11-2023-versailles/ by the AI Data Robotics Association that will be held in November 2023.
• “Confiance.ai Days” annual event organized by the Confiance.ai program, gathering conferences, roundtables, exhibition villages and results from all partners involved. The exact date of the event will be announced shortly (January 2024). The previous edition was held in 2022.