Core Concept

TESTMETRIX® organizes test data in a clear hierarchical structure.

A project forms the top level and can contain multiple pipeline definitions, multiple verification projects, and test systems. Each pipeline definition can be executed multiple times, resulting in pipeline runs, which in turn consist of individual test runs. A test run is composed of multiple test case executions.

A verification project can contain multiple requirement projects and test case projects.
Requirement projects contain the requirements of a specific area or test level, while test case projects contain the associated test case definitions.

Project
  │
  ├── Pipeline Definition
  │     ├── Pipeline Run
  │     │    ├── Test Run
  │     │    │    ├── Test Case Execution
  │     │    │    └── ...
  │     │    └── ...
  │     └── ...
  │
  ├── Verification Project
  │     ├── Requirement Project
  │     │    ├── High-Level Requirement
  │     │    ├── Low-Level Requirement
  │     │    └── ...
  │     │
  │     ├── Test Case Project
  │     │    ├── Test Case Definition
  │     │    └── ...
  │     │
  │     └── ...
  │
  └── Test System
        ├── DuT Component
        ├── ...
        ├── Test Equipment Component
        └── ...

Further Relations

In addition to the hierarchical structure, resources can be linked to each other across hierarchy boundaries.

1. A requirement project can be assigned to exactly one test case project and vice versa (1:1).
2. A test case definition can be linked to multiple low-level requirements. At the same time, a low-level requirement can be covered by multiple test case definitions (many-to-many).

The remaining relationships follow directly from the hierarchical structure shown above.

Short Description

Level	Description	Example
Project	Top-level unit, comparable to a GitLab project.	ESP32 Gateway
Pipeline Definition	Type of pipeline	- Post-Commit Pipeline that runs after every commit - or a Nightly Pipeline that runs every night - etc.
Pipeline Run	Concrete execution of a pipeline	Nightly Run #123
Test Run	Collection of executed test cases of a specific test suite	Integration Test, Unit Test, Smoke Tests, etc.
Test Case Execution	Individual test case execution within a test run	test_bluetooth_connection()
Verification Project	Functional structuring of requirements and test cases	Bluetooth Gateway Verification
Requirement Project	Grouping of functionally related requirements	Integration Requirements
High-Level Requirement	High-level functional requirement	Bluetooth Communication
Low-Level Requirement	Concretely testable technical requirement	Device must establish a Bluetooth connection within 30 seconds
Test Case Project	Grouping of functionally related test cases	Integration Tests
Test Case Definition	Description of a concrete test case	Verify Bluetooth pairing procedure
Test System	Test System configuration, consisting of DuT and test equipment components	Hardware-in-the-Loop ESP32-Gateway
DuT Component	Hardware or software component of the system under test	ESP32-Firmware, Backend-Software, etc.
Test Equipment Component	Hardware or software component used to test the DuT	Test Automation Framework, Tests, etc.

Connection between Verification and Test Execution

Test case definitions can be assigned to test executions via unique IDs.

This creates the following connection:

Test Case Execution
  → Test Case Definition
    → Low-Level Requirement

Based on this, TESTMETRIX® automatically calculates the requirement coverage for integration and system test levels.

Additional Test Data

Any type of additional data can be assigned to each test run and test case execution. This data is linked accordingly and displayed in the respective context.

Examples:

• Logs
• Metrics
• Screenshots
• Coverage Reports
• Test System Configurations
• Test Artifacts

Measurements are uploaded via a special JSON format and visualized as interactive charts.

Both linking and uploading happen primarily through the REST API, as TESTMETRIX® was built for seamless integration and full automation into existing systems.

For more details on the supported JSON format and how to upload metrics and test artifacts, see the API reference.

Was this page helpful?