KPI-Based Evaluation of L2/L3 ADAS: Transient Phases with Human Reference
Authors
Leandro Ronchi, Luca Veneroso, Alessio Anticaglia, Claudio Annicchiarico and Renzo Capitani.
Abstract
The growing diffusion of Advanced Driver Assistance Systems (ADAS) plays a crucial role in enhancing road safety and supporting the long-term objectives of Vision Zero, a strategy introduced in Sweden in the late 1990s and now adopted worldwide, which aims to eliminate fatalities and serious injuries from road traffic through safer vehicles, infrastructures, and driving behaviors.
In recent years, both academia and industry have devoted significant attention to the definition of Key Performance Indicators (KPIs) for the evaluation of ADAS performance. Most of the existing contributions, however, concentrate on steady-state metrics such as lateral offset, lateral oscillations, or stability margins, often developed within the framework of regulatory requirements such as UNECE R79, the United Nations regulation that defines uniform provisions for steering equipment, including Automatically Commandend Steering Function (ACSF), commonly known as Lane Centering (LC) systems, and Corrective Steering Function (CSF), commonly known as Lane Keeping Assist (LKA) systems.
While these indicators are essential for compliance and safety validation, they provide limited insight into how automated systems behave in transient maneuvers and how closely such behavior resembles that of human drivers, particularly with respect to comfort and acceptance.
In previous work, a KPI-based methodology was proposed to assess ADAS lane centering performance under steady-state conditions. The approach was validated across a virtual simulation environment and hardware-in-the-loop (HiL) platforms, showing its effectiveness in reducing development time and cost while supporting early-stage validation. Nevertheless, the analysis was restricted to straight-road scenarios, without the inclusion of a human driving reference and without addressing dynamic transitions, thus limiting its applicability to more realistic driving conditions.
In this sense, the present work can be seen as a natural extension of the previous study, further advancing it by focusing on crucial aspects such as the ability to accurately capture system behavior during transient maneuvers, as well as emphasizing driver comfort and user experience. Both features represent key enablers for fostering wider adoption of these systems.
The present study extends this methodology by introducing a new set of KPIs specifically tailored for transient maneuvers. These include the Aggressiveness Index, the Smoothness Index, and the Satisfaction Index, designed to capture the dynamic response of the ADAS and benchmark it against human driver behavior. The motivation is to evaluate not only the technical stability of the system, but also its ability to reproduce natural, human-like control actions.
From a human-centered perspective, there is a clear need for ADAS to evolve beyond simple regulatory compliance and technical robustness, towards design solutions that consider comfort, user acceptance, and alignment with driver expectations. In the literature and state of the art, strong attention is often given to driver subjectivity; however, maneuver evaluation is generally performed at an overall level through subjective questionnaires (QP), without a detailed and objective analysis of specific control actions.
To address this gap, we propose the definition of targeted Key Performance Indicators (KPIs) capable of objectively characterizing the behavior of specific transient maneuvers. These indicators enable the translation of the human’s subjective desired behavior into technical metrics, making it possible to develop and calibrate control systems that can not only reproduce human behavior in a natural and consistent way, but also achieve a superior performance while remaining aligned with the driver’s expectations and sensations.