Test Runs
A test run represents the results of one or more concrete test case executions.
Test runs are uploaded to TESTMETRIX® as JUnit XML files and are then visually processed.
Uploading Test Runs
Test runs are uploaded via the REST API.
At least one JUnit XML test report is transmitted.
Optional: Code Coverage
Additionally, a Cobertura XML file can be uploaded to analyze code coverage.
This is optional and mainly relevant for unit tests.
More information can be found in the corresponding section on code coverage.
Artifacts and Metrics
The focus of TESTMETRIX® is on simplifying root cause analysis by consolidating test data.
Therefore, additional data can be uploaded for every test run:
- Artifacts (e.g., logs, screenshots, files)
- Metrics (e.g., voltages, CPU or RAM utilization)
Test Case Level vs. Test Run Level
TESTMETRIX® distinguishes between two levels:
Test Case Level
Artifacts and metrics can be assigned directly to individual test cases.
This is particularly useful for test-case-specific measurements or individual events.
Test Run Level
Additionally, artifacts and metrics can be uploaded at the test run level.
This is suitable for data that is collected across the entire test run.
Examples:
- CPU utilization
- RAM utilization
- System metrics for embedded systems (e.g., Yocto)
These help identify system-wide issues and bottlenecks early.
Uploading Artifacts and Metrics
Artifacts and metrics are uploaded via the REST API.
Separate endpoints are available for this, which are described in more detail in the API reference.
Format of Measurement Data
Measurement data can be transmitted in the following formats:
- simple x,y value pairs
The same format applies to both the test case and test run levels.