A Real-World Latency Study of Microservice Architectures in Autonomous Driving

:::info Authors:

(1) Tobias Betz, Technical University of Munich, Germany;

(2) Long Wen, Technical University of Munich, Germany;

(3) Fengjunjie Pan, Technical University of Munich, Germany;

(4) Gemb Kaljavesi, Technical University of Munich, Germany;

(5) Alexander Zuepke, Technical University of Munich, Germany;

(6) Andrea Bastoni, Technical University of Munich, Germany;

(7) Marco Caccamo, Technical University of Munich, Germany;

(8) Alois Knoll, Technical University of Munich, Germany;

(9) Johannes Betz, Technical University of Munich, Germany.

:::

Table of Links

Abstract and I. Introduction

II. Related Work

III. Microservice Architecture for an Autonomous Driving Software

IV. Experiments

V. Results

VI. Discussion

VII. Conclusion, Acknowledgments, and References

\ Abstract—The automotive industry is transitioning from traditional ECU-based systems to software-defined vehicles. A central role of this revolution is played by containers, lightweight virtualization technologies that enable the flexible consolidation of complex software applications on a common hardware platform. Despite their widespread adoption, the impact of containerization on fundamental real-time metrics such as end-to-end latency, communication jitter, as well as memory and CPU utilization has remained virtually unexplored. This paper presents a microservice architecture for a real-world autonomous driving application where containers isolate each service. Our comprehensive evaluation shows the benefits in terms of end-to-end latency of such a solution even over standard bare-Linux deployment. Specifically, in the case of the presented microservice architecture, the mean end-to-end latency can be improved by 5-8%. Also, the maximum latencies were significantly reduced using container deployment.

I. INTRODUCTION

The automotive market is shifting towards software-defined vehicles (SDV), enabling a more software-centric automotive ecosystem. For example, the open-source consortium SOAFEE [1], [2] specifically targets SDV and brings together OEMs, Tier 1s, and chip manufacturers to work on the challenges. The E/E architecture of SDVs is based on a central computing unit in which a powerful high-performance computer manages and coordinates diverse functionalities. These functions encompass processing of sensor data, operation of infotainment systems, advanced driver assistance systems, and communication with external systems. This enables the separation of software from hardware functionality to achieve greater modularity and scalability. Lightweight virtualization techniques such as containerization enable efficient resource utilization and isolation of software components. This design philosophy empowers applications and services to operate independently within their dedicated virtual environments. When employing virtualization technologies in SDVs, stringent real-time criteria such as latencies and deadlines must be met. This concern becomes particularly important in autonomous vehicles, where the timely processing of sensor data within a predefined time window is critical for enabling prompt decision-making and control. Typically, end-to-end latencies of 100 ms are considered acceptable, wherein the sensor data must be swiftly processed, and the resulting output variables must be made available from the vehicle’s trajectory controller [3]. Therefore, the end-to-end latency directly impacts the vehicle’s ability to

\

\ navigate and respond to dynamic road conditions in a safe and reliable manner. Failure to meet these real-time requirements could lead to performance degradation and an increased risk of accidents [4]. In the context of software-defined autonomous driving architectures, practitioners have been experimenting with frameworks that simplify the difficult tasks of configuring, tuning, and optimizing the complex chains of architectural interdependencies. In particular, Autoware [5] and the Robot Operating System (ROS) 2 [6] are among the most widely used frameworks.

\ This paper introduces a microservice architecture—developed and applied to the research vehicle EDGAR [7]—that is explicitly designed for Autoware, an open-source autonomous driving software built on ROS 2. Since the impact of lightweight virtualization technologies on the latency of complex software has—to our knowledge—not yet been considered, this paper investigates the impact of containerization on the end-to-end latency in autonomous driving systems. Specifically, we focus on the end-to-end latency of a real-world autonomous driving architecture based on Autoware. We deploy the architecture on two different platforms (x86 and aarch64) using multiple configurations corresponding to an increasing level of container-based isolation (see Fig. 1). Using standard practice in industry [8], we use the container orchestration tool k3s [9] and Docker [10] to manage the functional dependencies among software packages and containers. Overall, the paper makes the following contributions:

\ • We present the structure and building process of a microservice architecture for autonomous driving software serving as a testbed for future work.

\ • We perform a comprehensive analysis of the impact of containerization using both specific benchmarks and direct measures on increasingly isolated microservice configurations.

\ • We quantitatively evaluate multiple real-time metrics, including end-to-end latency, jitter, system CPU and memory utilization.

\ Contrary to the common belief, our results show that containers can achieve lower end-to-end latency and better system utilization than bare Linux configurations. This underlines the challenge of finding the best-suited configuration options in very complex system scenarios and shows the benefit of containerization for future SDV systems. The developed microservice architecture will be contributed open-source to the Autoware Foundation (https://github.com/autowarefoundation/autoware).

II. RELATED WORK

Several papers discuss challenges and advancements in embedded systems and automotive software. Sax et al. [11] emphasize the shorter release cycles, increased variants, and software updates in modern vehicles. However, they do not provide any in-depth analysis of particular solutions or tools. The integration of new functionalities increases the complexity of vehicle systems, requiring careful considerations of the architecture and distribution of electronic control units to effectively manage this complexity [12]. Kugele et al. [13] discuss elastic service provisioning in intelligent vehicles. The management of different workloads, resource constraints, and changing user requirements is highlighted as a need. Hence, the importance of providing scalable and flexible services that are able to dynamically allocate resources and change performance characteristics based on real-time conditions.

\ Microservices and service-oriented architectures (SOA) have the potential to improve the flexibility of automotive systems. Lotz et al. [14] investigate the feasibility and impact of implementing a microservice architecture for driver assistance systems and demonstrate the reduction of complexity and improvement of software systems. Tamanaka et al. [15] present a conceptual framework for a fault-tolerant architecture and highlight the use of microservices and containerization as critical components. In [16], a literature review explores design principles and architectural refinement strategies for microservices. Through a systematic mapping study, Kukulicic et al. [17] analyzes the adoption of SOA in automotive software. Functional usability stands out as the most relevant benefit, while issues such as e.g., security, safety, and reliability are identified as challenges. Previous research provides overviews, on a purely theoretical basis, of the challenges and benefits of moving to a microservice architecture (see [18]).

\ Regarding the performance impact of virtualization and containerization technologies on diverse systems, in [19] the authors introduce a benchmarking suite to assess the resource costs of various virtualization technologies. They compare the performance of hardware native hypervisors, hosted hypervisors, and containers using reference benchmarks. Morabito et al. [20] focus on evaluating the performance of containerization on Internet-of-Things edge environments. The strengths and weaknesses of various low-power devices when dealing with container-virtualized instances are highlighted. Notably, both demonstrate that virtualized or containerized systems show acceptable performance compared to bare-metal systems. Felter et al. [21] compare the performance of virtual machines (VMs) and Linux containers within cloud computing environments. The study demonstrates that containers outperform or match the performance of VMs in most cases, emphasizing the potential benefits of using containers in cloud architectures. Similarly, in [22], the authors conduct research on container-based virtualization in high-performance computing, highlighting the low overhead and potential for nearnative performance. While all these studies provide valuable insights, they lack experiments on real-world use cases. In the automotive area, Rajan et al. [23] explore the technique of bringing virtualization into automotive multicore controllers. The authors evaluate the performance of a virtualized system in terms of core loading, interrupt timing, and task timing parameters. Long et al. [24] develop a general benchmark that yields results consistent with the conclusions mentioned previously. Furthermore, they specifically focus on the startup time of microservice-based Autoware automotive applications demonstrating that virtualization and containerization are suitable and viable options. The adoption of these virtualization technologies might be beneficial in the automotive industry. For the development of robot software, ROS 2 is the most widespread framework. [25] presents an exemplary architecture tailored to autonomous driving and the possibilities of using it for high-speed autonomous racing are presented in [26]. The proposed racing architecture is based on microservices where each functional module, e.g., perception, planning, and control, is deployed as a container. Autoware [5] represents the most comprehensive open-source initiative dedicated to ROS 2 for autonomous driving software. In the literature, there are already several different frameworks [27], [28], [29], [30] that allow to measure ROS 2 applications. These are normally based on ros2_tracing [31], which instruments trace points into the middleware accordingly. This allows determining callback times as well as end-to-end latencies. In addition, there are benchmark tools that are mostly limited to simple examples where statements can be made about the DDS latency [32], [33] or the entire system performance [34]. For a single-threaded executor system, timing analysis is performed in [35]. Reke et al. [25] conduct an end-to-end latency and corresponding jitter analysis for an entire application. In [36] an analysis for Autoware is realized, with a focus on embedded hardware. The authors in [37] analyze the impact of the different system abstraction layers on the end-to-end latency of Autoware. They also tried different Linux scheduling configurations to improve the timing behavior on bare-metal systems. In [38], [39] the focus is only on the influence of the DDS layer. For the ROS 2 autonomous racing software presented in [26], a latency evaluation is carried out in [4]. The focus is on the application layer and vehicle stability impact of timing. However, the influence of the microservice architecture is not evaluated. Based on the current state of the art, it is impossible to find an assessment of the impact of containerization and a corresponding microservice architecture on the end-to-end latency of ROS 2 applications.

\

:::info This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license.

:::

\