Circuit Layer Operations Per Second (CLOPS) benchmark
The CLOPS protocol [1] assesses the execution speed and reliability of a quantum computer in running a series of parameterized square quantum circuits. The original formulation [1] introduces the \(\mathrm{CLOPS_v}\) figure of merit, while a subsequent update to the protocol [2] presented in 2023 defines an alternative figure of merit \(\mathrm{CLOPS_{h.v0}}\). This subsequent figure of merit has then been lately formalized as \(\mathrm{CLOPS_h}\) in 2026 in [2] with differences compared to \(\mathrm{CLOPS_{h.v0}}\). These figures of merit are reported in the results table for completeness. Reader may notice that both \(\mathrm{CLOPS_{h.v0}}\) and \(\mathrm{CLOPS_h}\) are referred to as \(\mathrm{CLOPS_h}\) in the literature. However, the two protocols differ in several aspects, which is why we distinguish between their definitions here.
List of acronyms
CE: Constructor Evaluation (checked if the evaluation is done by the chip manufacturer)
SP: Scientific paper (checked if a scientific paper explain the results)
M: number of templates
K: number of different circuits for each template
QV layers: number of Quantum volume layers (equivalent to \(\log_2 (QV)\))
| Date & Ref | Company | CE | SP | M | K | Shots | QV layers | CPU / QPU | Technology | #qubits | \(\mathrm{CLOPS_v}\) | \(\mathrm{CLOPS_{h.v0}}\) | \(\mathrm{CLOPS_h}\) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2021/10 [3] | IBM | x | x | 100 | 10 | 100 | 5 | ibmq_bogota | Superconducting | 5 | 1 419 | ||
| 2021/10 [3] | IBM | x | x | 100 | 10 | 100 | 5 | ibmq_toronto | Superconducting | 27 | 951 | ||
| 2021/10 [3] | IBM | x | x | 100 | 10 | 100 | 5 | ibmq_brooklyn | Superconducting | 65 | 753 | ||
| 2022/02 [4] | Rigetti | x | 100 | 10 | 100 | Aspen-11 | Superconducting | 40 | 844 | ||||
| 2022/02 [4] | Rigetti | x | 100 | 10 | 100 | Aspen-M | Superconducting | 80 | 892 | ||||
| 2023/08 [5] | QuTech | x | 100 | 10 | 100 | 2 | Starmon-5 | Superconducting | 5 | 372 | |||
| 2025/02, 2026/09 [6][7] | IBM | x | 100 | 10 | 100 | 9 | ibm_marrakesh (Heron r2) | Superconducting | 156 | 2 400 | 195 000 | 300000 | |
| 2025/02, 2026/09 [6][7] | IBM | x | 100 | 10 | 100 | 9 | ibm_fez (Heron r2) | Superconducting | 156 | 2 400 | 195 000 | 300000 | |
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 9 | ibm_torino (Heron r1) | Superconducting | 156 | 2 400 | 210 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_sherbrooke (Eagle r3) | Superconducting | 127 | 2 400 | 30 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_kyiv (Eagle r3) | Superconducting | 127 | 2 400 | 30 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_brisbane (Eagle r3) | Superconducting | 127 | 2 400 | 180 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_quebec (Eagle r3) | Superconducting | 127 | 2 400 | 32 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_kawasaki (Eagle r3) | Superconducting | 127 | 2 400 | 29 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_brussels (Eagle r3) | Superconducting | 127 | 2 400 | 220 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_rensselaer (Eagle r3) | Superconducting | 127 | 2 400 | 32 000 | ||
| 2025/02 [6] | IBM | x | 100 | 10 | 100 | 7 | ibm_strasbourg (Eagle r3) | Superconducting | 127 | 2 400 | 220 000 | ||
| 2024/08 [8] | IQM | x | x | 100 | 10 | 100 | 5 | Garnet | Superconducting | 20 | 2 600 | ||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_phoenix (Nighthawk r2) | Superconducting | 120 | 2000000 | ||||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_kingston (Heron r2) | Superconducting | 156 | 340000 | ||||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_berlin (Nighthawk r1) | Superconducting | 120 | 25000 | ||||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_aachen (Heron r3) | Superconducting | 156 | 330000 | ||||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_miami (Nighthawk r1) | Superconducting | 120 | 24000 | ||||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_boston (Heron r3) | Superconducting | 156 | 340000 | ||||
| 2026/09 [7] | IBM | x | 100 | 10 | 100 | ibm_pittsburgh (Heron r3) | Superconducting | 156 | 330000 |
Motivation
The Circuit Layer Operations Per Second (\(\mathrm{CLOPS_v}\)) [3] was introduced by IBM in 2021. CLOPS is designed to simultaneously capture three critical aspects of quantum computation: quality, speed, and circuit scale. Specifically, the \(\mathrm{CLOPS_v}\) quantifies the number of Quantum Volume (QV) circuits [1] that a quantum processor can execute reliably per unit of time.
\(\mathrm{CLOPS_{h.v0}}\) is the variation of the \(\mathrm{CLOPS_v}\) protocol presented in 2023 and relies on the EPLG protocol rather that the Quantum Volume.
A third version called \(\mathrm{CLOPS_{h}}\) has been formalized in 2026 in [9].
Protocol
The initial protocol detailed in [3] is based on the Quantum Volume protocol and defines \(M\) template circuits. Each template circuit is composed of \(K\) different real quantum circuits with a set of parameter \(\vec{\theta}_i\) used to initialize each \(SU(4)\) gate. The permutation layers \(\pi_i\) are the same for all the circuits generated from a single template (see Fig. 1).
The intial protocol introduced in [3] defines the figure of merit \(\mathrm{CLOPS_v}\) as:
\[\mathrm{CLOPS_v} = \frac{M \times K \times S \times D}{\mathrm{time\_taken}}\]where \(M\) is the number of templates, \(K\) is the number of simulations done for each template with different parameters for the layers \(SU(4)\), \(S\) is the number of shots and \(D=\log_2 QV\) layers. The initial protocol sets \(M=100\) and \(K=10\), which leads to the execution of \(1000\) different quantum circuits. Each circuit is run with 100 shots. The \(\mathrm{time\_taken}\) accounts for the total execution time including data transfer to the QC, transpilation of high-level quantum gates to low-level control pulses, qubit measurement and reset time as well as inter-circuit delays.
Assumptions
- The compilation of each template circuit (high-level compilation and circuit optimization) is not accounted in the measured \(\mathrm{time\_taken}\).
- For each template circuit, ten parameterized circuits are generatacross all the pairs of qubits. This initied. The parameters \(\vec{\theta}_i\) for the \(i\)-th circuit are only instantiated once the execution of the \((i-1)\)-th circuit has been completed.
- Multiple template circuits may be run in parallel on the quantum chip.
- It is essential to specify the quantum volume used in the benchmark to interpret the results. Indeed, a lower quantum volume might lead to a higher CLOPS score.
Limitations
- The CLOPS score comparison is only meaningful if the same quantum volume is used for all the experiments (this is due to the use of the logarithm of the quantum volume in the computation of the CLOPS score).
- The limits of the quantum volume (e.g., scalability) also apply to the metric \(\mathrm{CLOPS_v}\).
Update of the CLOPS protocol ( \(\mathrm{CLOPS_{h.v0}}\) ) - 2023/11
An update concerning the measurement of CLOPS values has been proposed in [2] with a new protocol measuring \(\mathrm{CLOPS_{h.v0}}\) (\(\mathrm{h}\) meaning hardware). This new protocol changes how layers are considered. In the initial protocol, a layer was defined by a random permutation of all the qubits followed by two-qubit gates (from the \(SU(4)\) group) across all the pairs of qubits. This initial protocol was favorable for fully connected topologies, which could more easily implement the random permutation than for topologies with restricted connectivity. The new protocol \(\mathrm{CLOPS_h}\) now relies on the EPLG protocol instead of the quantum volume. It splits sequences of gates that cannot be run in parallel into sublayers, artificially inflating the overall number of layers, which tends to increase the value of \(\mathrm{CLOPS_h}\) compared to \(\mathrm{CLOPS_v}\) even without any hardware improvement. It has the effect of inflating the CLOPS score for quantum computers that are sparsely connected.
Update of the CLOPS protocol (\(\mathrm{CLOPS_h}\)) - 2026/08
Another update concerning the measurement of CLOPS values has been formally presented in [9].
Implementations
An implementation of the \(\mathrm{CLOPS_v}\) based on the quantum volume developed by IQM is available here.
The \(\mathrm{CLOPS_{h.v0}}\) metric relies on the EPLG protocol. At the time of writing, an implementation of the complete \(\mathrm{CLOPS_h}\) protocol was not found.
References
- [1]A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, “Validating quantum computers using randomized model circuits,” Physical Review A, vol. 100, no. 3, p. 032328, 2019.
- [2]A. Wack and D. McKay, “Updating how we measure quantum quality and speed.” November 2023. Available at: https://www.ibm.com/quantum/blog/quantum-metric-layer-fidelity. [Accessed: February 26, 2025]
- [3]A. Wack et al., “Quality, Speed, and Scale: three key attributes to measure the performance of near-term quantum computers.” 2021. Available at: https://arxiv.org/abs/2110.14108
- [4]Rigetti, “Rigetti Computing Announces Commercial Availability of 80-Qubit Aspen-M System and Results of CLOPS Speed Tests.” 2022. Available at: https://investors.rigetti.com/node/7066/pdf. [Accessed: February 15, 2022]
- [5]W. van der Schoot, R. Wezeman, P. T. Eendebak, N. M. P. Neumann, and F. Phillipson, “Evaluating three levels of quantum metrics on quantum-inspire hardware,” arXiv preprint arXiv:2310.01120, 2023.
- [6]I. B. M. quantum, “Quantum processing units.” February 2025. Available at: https://quantum.ibm.com/services/resources. [Accessed: February 26, 2025]
- [7]I. B. M. quantum, “Compute resources.” 2026. Available at: https://eu-de.quantum.cloud.ibm.com/computers. [Accessed: September 19, 2026]
- [8]L. Abdurakhimov et al., “Technology and Performance Benchmarks of IQM’s 20-Qubit Quantum Computer,” arXiv preprint arXiv:2408.12433, 2024.
- [9]A. Wack, “CLOPS: Benchmarking System Speed at Utility Scale,” arXiv preprint arXiv:2608.18044, 2026.