Test System Configuration
Test system configurations are a central component of TESTMETRIX®.
In a test system, the hardware components – such as test equipment and the Device under Test (DuT), i.e. the unit under test – are typically stationary.
The software, especially on the DuT, changes with every test run, as new software or firmware versions are continuously tested during development.
A DuT often consists of multiple components (e.g. multiple PCBs). Depending on the complexity of the test system, there are thus multiple hardware components, each with their own changing software versions.
Traceability through Test System Configurations
To ensure complete traceability, the software versions used are captured for every test run via the test system configuration.
This ensures you always know which software version ran on which component.
Benefit for Root Cause Analysis
Defining a test system with all its associated hardware and software components ensures high reproducibility. This significantly simplifies root cause analysis because:
- Test environments can be reproduced exactly
- Differences between test runs become visible
- Errors can be specifically attributed to certain software versions
Using the Test System Configuration
1. Define the Test System
The test system is defined once in the user interface.
A distinction is made between:
- Device under Test (DuT)
- Test equipment (e.g. oscilloscopes, logic analyzers, test frameworks, test suites, etc.)
Third-party test equipment, such as oscilloscopes, typically has a fixed software or firmware version, so versioning it is rarely necessary. DuT components and in-house developed test equipment, on the other hand, typically have changing software versions that need to be recorded.
This distinction is important for the later test system configuration file.
2. Use the Test System Configuration File
After creating the test system via the user interface, a template for a test system configuration file is available for download.
This file tells TESTMETRIX® which software versions ran on each component after every test run.
3. Update Software Versions
After each test run, the corresponding fields in the test system configuration file are automatically filled (e.g. via script) with the actually used software versions.
4. Upload via REST API
Subsequently, the test system configuration file is uploaded together with the test data via the REST API.
Example: Bluetooth Gateway Test System
The following test system consists of an ESP32 Gateway as the DuT (Device under Test) and the test equipment required for test execution. The test equipment includes a Bluetooth Peripheral Emulator, a TAF (Test Automation Framework), and the Bluetooth Test Suite.
After defining this test system in the user interface, the following template configuration file is available for download:
testsystem_config:
testsystem_name: "Bluetooth Gateway Test System"
components:
- name: "ESP32 Gateway"
sw_version: ""
- name: "Bluetooth Peripheral Emulator""
sw_version: ""
- name: "TAF"
sw_version: ""
- name: "Bluetooth Test Suite"
sw_version: ""
The sw_version fields are intentionally left empty. Before each test run, fill them with the actually used software versions and then upload the file together with the test data via the REST API.