Core Concept

TESTMETRIX® organizes test data in a clear hierarchical structure.

A project forms the top level and can contain both multiple pipeline definitions and multiple verification projects. Each pipeline definition can be executed multiple times, resulting in pipeline runs, which in turn consist of individual test runs. A test run is composed of multiple test case executions.

A verification project can contain multiple requirement projects and test case projects.
Requirement projects contain the requirements of a specific area or test level, while test case projects contain the associated test case definitions.

Project
  │
  ├── Pipeline Definition
  │     ├── Pipeline Run
  │     │    ├── Test Run
  │     │    │    ├── Test Case Execution
  │     │    │    └── ...
  │     │    └── ...
  │     └── ...
  │
  ├── Verification Project
  │     ├── Requirement Project
  │     │    ├── High-Level Requirement
  │     │    ├── Low-Level Requirement
  │     │    └── ...
  │     │
  │     ├── Test Case Project
  │     │    ├── Test Case Definition
  │     │    └── ...
  │     │
  │     └── ...
  │
  └── ...

Level	Description	Example
Project	Top-level unit, comparable to a GitLab project.	ESP32 Gateway
Pipeline Definition	Type of pipeline	- Post-Commit Pipeline that runs after every commit - or a Nightly Pipeline that runs every night - etc.
Pipeline Run	Concrete execution of a pipeline	Nightly Run #123
Test Run	Collection of executed test cases of a specific test suite	Integration Test, Unit Test, Smoke Tests, etc.
Test Case Execution	Individual test case execution within a test run	test_bluetooth_connection()
Verification Project	Functional structuring of requirements and test cases	Bluetooth Gateway Verification
Requirement Project	Grouping of functionally related requirements	Integration Requirements
High-Level Requirement	High-level functional requirement	Bluetooth Communication
Low-Level Requirement	Concretely testable technical requirement	Device must establish a Bluetooth connection within 30 seconds
Test Case Project	Grouping of functionally related test cases	Integration Tests
Test Case Definition	Description of a concrete test case	Verify Bluetooth pairing procedure

Connection between Verification and Test Execution

Test case definitions can be assigned to test executions via unique IDs.

This creates the following connection:

Test Case Execution
  → Test Case Definition
    → Low-Level Requirement

Based on this, TESTMETRIX® automatically calculates the requirement coverage for integration and system test levels.

Additional Test Data

Any type of additional data can be assigned to each test run and test case execution. This data is linked accordingly and displayed in the respective context.

Examples:

• Logs
• Metrics
• Screenshots
• Coverage Reports
• HiL Configurations
• Test Artifacts

Measurements are uploaded via a special JSON format and visualized as interactive charts.

Both linking and uploading happen primarily through the REST API, as TESTMETRIX® was built for seamless integration and full automation into existing systems.

For more details on the supported JSON format and how to upload metrics and test artifacts, see the API reference.

Was this page helpful?