Metriq benchmark suite
The Metriq Benchmark Suite [1] was introduced in March 2026 by the Unitary Foundation, USA, as part of the Metriq platform. The initiative is also supported by the Open Quantum Benchmark Committee (OQBC), established in 2024 to facilitate development and adoption of quantum benchmarks.
Motivation
The Metriq Benchmark Suite aims to establish an open-source, community-driven framework for evaluating quantum computing performance. By enabling researchers and developers to contribute benchmarking protocols, execute them across different hardware platforms, and share results publicly, it seeks to improve the transparency, reproducibility, and comparability of quantum computing benchmarks.
Architecture
The Metriq platform follows a modular architecture composed of three independent components:
- metriq-gym — Benchmark execution: A Python-based runner providing standardized implementations of benchmarking protocols, configurable parameters, hardware abstraction, and automated job execution. It uses the qBraid SDK for interoperability across quantum hardware providers.
- metriq-data — Data collection: A version-controlled repository containing structured, schema-validated JSON benchmarking results. Records include the device, provider, benchmark version, configuration, and execution timestamp, enabling traceability and independent analysis.
- metriq-web — Results visualization: A public web interface that aggregates, processes, and presents benchmarking results, supporting comparisons across devices and protocols.
The platform enables contributors to implement new protocols through a community review process, execute them using a common interface, and publish their results in a shared dataset. Benchmark configurations are defined through explicit parameter schemas to improve experimental reproducibility.
Benchmark instances
The suite contains eight benchmarking protocols organized into two categories described in the following table.
| Benchmark | Category | Instance size | Additional configuration |
|---|---|---|---|
| Bell State Effective Qubits (BSEQ) | System | All available connected qubit pairs | 1,000 shots per circuit |
| Error Per Layered Gate (EPLG) | System | 10, 20, 50, 100 qubits | 10 samples, 1,000 shots |
| Mirror Circuits | System | 8, 16, 24, 32, 64, 128 qubits | Variable circuit depths, 10 circuit samples, 1,000 shots |
| CLOPS | System | 100 qubits | 100 layers, 1,000 circuits, 100 shots per circuit |
| QML Kernel | Application | 10, 20, 30, 50 qubits | 1,000 shots per circuit |
| Linear Ramp QAOA (LR-QAOA) | Application | 10, 20, 50, 100 qubits | Weighted Max-Cut on 1D chains, 10 QAOA layers |
| Wasserstein Indicator Test (WIT) | Application | 7 qubits | Fixed wormhole-inspired teleportation circuit, 8,192 shots |
| Quantum Fourier Transform (QFT) | Application | 4, 8, 12, 20 qubits | Up to 3 circuits per size, 1,000 shots |
These protocols are selected to cover different performance characteristics while keeping execution costs manageable The listed configurations correspond to the experiments reported in the publication. Benchmark parameters are configurable, and individual protocols can support additional instance sizes or circuit variants.
The figure of merit used to evaluate the performance of the quantum computer is the Metric Score (MS), where normalized benchmark scores are combined using a weighted average across the different instances and benchmarks.
Devices being benchmarked
The composite-score evaluation reported in the publication includes 11 quantum processors from five hardware providers.
| Provider | Device | Technology | Qubits |
|---|---|---|---|
| IBM | ibm_boston | 156 | Superconducting |
| IBM | ibm_pittsburgh | 156 | Superconducting |
| IBM | ibm_kingston | 156 | Superconducting |
| IBM | ibm_marrakesh | 156 | Superconducting |
| IBM | ibm_fez | 156 | Superconducting |
| IBM | ibm_torino | 133 | Superconducting |
| Quantinuum | quantinuum_h2-2 | 56 | Trapped ions |
| IQM | iqm_emerald | 54 | Superconducting |
| IQM | iqm_garnet | 20 | Superconducting |
| Rigetti | rigetti_ankaa-3 | 82 | Superconducting |
| OriginQ | wukong-72 | 72 | Superconducting |
Limitations
The Metriq Benchmark Suite has several limitations, some of which are explicitly acknowledged by the authors:
-
Limited hardware scope: The current suite primarily targets NISQ gate-based quantum computers. Analog quantum computing platforms and fault-tolerant logical-qubit systems are not yet covered by dedicated benchmarking protocols.
-
Incomplete cross-platform comparability: The authors explicitly identify inconsistencies in the availability and reporting of device and job metadata across hardware providers. In particular, CLOPS requires reliable QPU execution-time information, which is not exposed by all providers.
-
Metriq score limitation: The aggregation of benchmark results introduces additional limitations, discussed in the dedicated Metriq Score page.
Implementation
The Metriq Benchmark Suite is implemented through three open-source repositories maintained by the Unitary Foundation: metriq-gym, metriq-data, and metriq-web.
Benchmark results and device comparisons are publicly accessible through the Metriq platform.
References
- [1]A. Cosentino et al., “Metriq: A Collaborative Platform for Benchmarking Quantum Computers,” arXiv preprint arXiv:2603.08680, 2026.