Autonomous driving is a vision for the future that offers numerous benefits, such as enabling vehicles to respond dynamically to complex situations and improving the safety of road transport. However, it is also associated with significantly higher demand on driver-assistance systems (DAS) and the associated software and sensors, which in turn increases the complexity of the vehicle as a whole. In the future, between 300 and 500 million lines of software code will be required to make autonomous cars a reality. [1] Even today, cars feature a network of high-performance computers (HPC) as well as over 100 control units, which can exchange up to 25 gigabytes of data between them within just one hour. [2,3] Communication within the vehicle is ensured by around 14 different bus systems and transmission standards. [4] All of these factors combine to present an enormous challenge for validation – especially for the validation of control devices associated with the domain architecture for driver assistance systems (DAS). This is because ADAS/AD systems present challenges that are partially controltechnology based and partially algorithmic in nature, such as longitudinal control, sensor-data fusion, object classifications and the use of artificial intelligence. Control units in these domains need to be able to communicate with each other in high volumes as they are tasked with making safety-critical designs regarding the vehicle’s responses in road traffic. Validating this type of software often requires scenario-based testing in complex co-simulation environments. In addition, it is not possible to specify all of the countless parameters in all possible scenarios, which further complicates validation.
Focusing on DAS and the network of primary control units
The ASAP Group has therefore developed a test concept for the seamless validation of vehicle electronics, with particular focus on the network of high-performance computers (HPCs) and DAS functions. ASAP offers its customers a full range of services, from designing, commissioning and operating testing systems to requirements analysis, test specification and test automation through to reporting and final approval recommendations. In terms of the test infrastructure, ASAP Test Systems specialists started by borrowing the design for complete-vehicle HIL systems. These systems map all vehicle electronics and how they are networked, with the HPCs and an average of 60 further control units installed as real components in the test system. Correctly wiring communications links between all control units in the test system presents a particular challenge at this juncture. Only specific components that are still under development or have not yet been equipped with compatible software are simulated. So, unlike in component HILs, there is little need for restbus simulation as all control units are present. This reduces the test scope: complete-vehicle HIL system testing focuses on the integration of event chains – i.e. ensuring that all control units communicate faultlessly – because all components from sensor input onwards are physically available in the test system. Using complete-vehicle HIL systems therefore offers fewer opportunities for manipulation, with the test moving away from the signal level towards driver control. It only simulates the input driving scenarios and surroundings and the driver’s reactions. All internal signals and interfaces between the control units have to be issued independently by the control units in the test system. ASAP therefore integrates an additional steering table into the test benches so that its test engineers can make manipulations at the driver level. This makes it possible to input the driver’s reactions manually, thereby simulating the steering forces from the driver. This input is then fed into the test system via special interfaces.
Complex driving functions require complex test systems
Given the inherent complexity and the number of functions to be tested, complete-vehicle GIL systems are subject to tough requirements. On the one hand, flashing control devices in complete-vehicle HIL systems represents a major challenge: test engineers must ensure that all components have the latest software version at all times. During these updates, it is important to bear in mind the interdependencies between different control units. On the other hand, developers need to ensure that the test system – and the sensors used in the virtual complete-vehicle – receive information during virtual test drives, such as that another road user is braking a few meters ahead. Consequently, the biggest challenge for developing a complete-vehicle HIL system is the timing. If calculations and tests are to be accurate and meaningful, the models must provide a coherent overall picture for the vehicle’s environmental sensors. For all sensor technology to combine simultaneously – and produce a coherent scenario – the test bench must provide all information with synchronized timing (i.e. deterministically). If, for example, a motorway pilot system is validated with a simulated test drive at 130 kph, with a slower-moving car merging into the lane ahead from the inside lane, all environmental data such as speed, distances and road layout must be fed into the control unit simultaneously and as a complete image in order to facilitate coherent data fusion. Only then, in conjunction with all other control units in the event chain, can the correct function in this example be identified, namely initiating braking. A further challenge is that, in the test system, the sensors must feed information on the recorded environment to the control units, which means this has to be structured as a “closed-loop system”. This term describes a situation in which the real, installed network of control units and computers interacts with the simulated environment. If, for example, the control units accelerate, the simulated environment must change accordingly and provide suitable feedback to the control units and sensors. Additionally, feedback on the environment must be transmitted directly from the sensors to the control unit, without any detours, so that the control unit can react immediately to the situation. So, to simulate the situation to the control units directly, the real sensors must be disconnected from the control units.
Validating dynamic processes through scenario-based testing
After constructing complete-vehicle HIL systems, experts in ASAP’s Testing and Integration team ensure they are commissioned correctly. In parallel with setting up the test systems, they also conduct the requirements analyses and test case specification and automation required for the test infrastructure. ASAP does this using the scenario-based testing method. By referring to the PEGASUS project, ASAP can conduct effective and efficient testing while simultaneously taking heed of the risks involved. The PEGASUS research project, pursued by the OEMs and numerous partners from the worlds of business and science, aims to establish “generally accepted quality criteria, tools and methods as well as scenarios and situa-tions for the release of highly automated driving functions” [5] and thus accelerate the realisation of autonomous driving. ASAP looked at the results of the PEGASUS research results while adapting scenario-based testing for complete-vehicle HIL system validation – and thereby reduced the complexity inherent in validation due to the near-infinite number of possible test cases. In contrast to requirements-based testing, scenario-based testing can also be used to test dynamic processes – such as speed changes and various traffic situations with a variable environment and different ambient conditions.
The ASAP specialists responsible for test design specify both the required scenarios and the test cases. In terms of scenario specification, the ASAP experts firstly define in detail all static and dynamic objects to be included in a scenario, including environmental data and parameter spaces. This includes, for instance, all possible distances and speeds of a vehicle driving ahead. By contrast, in the subsequent step of specifying test cases, the level of abstraction is far lower. Instead, test drives are set out with specific values for all objects involved in the test case so that (for example) the overtaking manoeuvre described as a scenario can be carried out correctly. Once the driving scenarios and test procedures have been defined, including the expected outcomes (test cases), ASAP then moves on to validating the E/E infrastructure for the complete-vehicle HIL systems.
Automated test case generation with keyword-driven testing
Although scenario-based testing makes it considerably easier to carry out testing, the vast number of different scenarios adds to the complexity of test automation. The solution is to use test automation to depict thousands of test cases in such a way that the test themselves can be fully automated. For this to be pos-sible, the descriptions of test cases and driving scenarios must be automatically implemented in the corre-sponding tools. Test automation also has to bring the entire toolchain together and ensure that all tools engage with each other automatically and without interruption. In order to implement test cases more swiftly and reduce the complexity of test automation, ASAP combines scenario-based testing with keyword-driven testing for use in the validation of complete-vehicle HIL test systems. In this form of test case description, which is certified in accordance with ISO 29119-5, individual test steps are stored in a database in both human-readable and machine-readable formats. This means that ASAP starts by writing a corre-sponding script for each defined test step – the so-called keywords – so that it can be carried out automatically. A test step might be a command to perform a specific driver action. All finally defined keywords (test steps) are universally applicable and can be parametrized in the database. This creates reusable test steps that only need to be parametrised with different input values. The process of inputting test steps to create a test case is also automated. The result? Partial automation of test automation, which delivers immense time savings due to the enormous volume of required test cases. In addition, in the event of changes, the corresponding keywords only has to be amended once in the central database – with any changes automatically integrated into all test cases. A further benefit is that test steps are stored in human-readable and machine-readable formats in the database. This means, for example, that real test drives can be reproduced using the documented data and repeated virtually any number of times until the scenario has been checked to ensure consistent quality. So, with this combination of scenario-based and keyword-driven testing, ASAP reduces the complexity and effort involved in test preparation and execution, ultimately ensuring time-saving, cost-effective and comprehensive validation at its test centre.
References
[1] Roland Berger: Global Automotive Supplier Study
[2] Autos brauchen neue Architekturen!: https://www.se-trends.de/autos-brauchen-neue-architekturen/
[3] Roland Berger: Consolidation in vehicle electronic architectures
[4] Wie Ethernet das Auto verändert: https://www.elektroniknet.de/automotive/wie-ethernet-das-auto-veraendert.150788.html
[5] PEGASUS research project. Securing automated driving effectively: www.pegasusprojekt.de/en/about-PEGASUS