Featured Dataset
Quantum Flakines Dynamic Testing
Detect flaky tests in quantum software using Dynamic approach Abstract This repository is a companion page for the paper “Detecting Flakiness in Quantum Software: A Dynamic Testing Approach.” The dataset contains
File Size57.7 MB
Data TypeDataset
Published2026
Licensecc-by-4.0
Total Views290
Total Downloads3,175
Detect flaky tests in quantum software using Dynamic approach
Abstract
This repository is a companion page for the paper “Detecting Flakiness in Quantum Software: A Dynamic Testing Approach.”
The dataset contains two types of files:
1. Per-release execution-summary CSV files: 23 CSV files, one for each Qiskit Terra release studied. Each file summarizes the outcomes of 10,000 executions of the test suite for that release.
2. Root-cause and fix-labels file: one spreadsheet containing manually assigned root-cause and fix-pattern labels for the flaky tests identified in the study.
Per-release CSV files
Each of the 23 CSV files shares the following common schema.
| Attribute | Description |
| test_case_name | The identifier of the individual. |
| passed_attempts | Number of runs in which the test passed. |
| failed_attempts | Number of runs in which the test failed. |
| skipped_runs | Number of skipped runs (e.g., via skip markers or environment constraints);skips were outside our control. |
| total_runs | The total number of executions attempted for the test case; computed as passed_attempts + failed_attempts + skipped_runs; equals 10,000 in all retained rows. |
| success_rate | The fraction of successful executions; computed as passed_attempts/total_runs. |
| qiskit_version | The terra release tag used for the run. |
| python_version | Python version used (always 3.8). |
| rust_version | Rust toolchain version used to satisfy terra build requirements for that release. |
Root-cause and fix-labels file
In addition to the per-release execution summaries, the dataset includes a spreadsheet named
qiskit_flaky_tests_root_cause_fix_labels.xlsx.This file contains the manually assigned root-cause and fix-pattern labels for the flaky tests identified in the study. Each row corresponds to one flaky test case and records the last release in which the test was observed to be flaky, whether and when it was fixed or removed, and the manually assigned root-cause and fix-pattern categories.
The schema is as follows
Attribute | Description |
test_case_name | Fully qualified identifier of the flaky test case. Each entry is hyperlinked to the corresponding GitHub sou 📤 Share this pageFound this useful? Share it with your network. Files are hosted on the source repository. Click download to access the full dataset. Dongchan, Kim (2026). Quantum Flakines Dynamic Testing. https://doi.org/10.5281/zenodo.20500432
|