Circuit Layer Operations Per Second (CLOPS) benchmark

The CLOPS protocol [1] assesses the execution speed and reliability of a quantum computer in running a series of parameterized square quantum circuits. The original formulation [1] introduces the \(\mathrm{CLOPS_v}\) figure of merit, while a subsequent update to the protocol [2] presented in 2023 defines an alternative figure of merit \(\mathrm{CLOPS_{h.v0}}\). This subsequent figure of merit has then been lately formalized as \(\mathrm{CLOPS_h}\) in 2026 in [2] with differences compared to \(\mathrm{CLOPS_{h.v0}}\). These figures of merit are reported in the results table for completeness. Reader may notice that both \(\mathrm{CLOPS_{h.v0}}\) and \(\mathrm{CLOPS_h}\) are referred to as \(\mathrm{CLOPS_h}\) in the literature. However, the two protocols differ in several aspects, which is why we distinguish between their definitions here.

List of acronyms
CE: Constructor Evaluation (checked if the evaluation is done by the chip manufacturer)
SP: Scientific paper (checked if a scientific paper explain the results)
M: number of templates
K: number of different circuits for each template
QV layers: number of Quantum volume layers (equivalent to \(\log_2 (QV)\))

Date & Ref Company CE SP M K Shots QV layers CPU / QPU Technology #qubits \(\mathrm{CLOPS_v}\) \(\mathrm{CLOPS_{h.v0}}\) \(\mathrm{CLOPS_h}\)
2021/10 [3] IBM x x 100 10 100 5 ibmq_bogota Superconducting 5 1 419
2021/10 [3] IBM x x 100 10 100 5 ibmq_toronto Superconducting 27 951
2021/10 [3] IBM x x 100 10 100 5 ibmq_brooklyn Superconducting 65 753
2022/02 [4] Rigetti x 100 10 100 Aspen-11 Superconducting 40 844
2022/02 [4] Rigetti x 100 10 100 Aspen-M Superconducting 80 892
2023/08 [5] QuTech x 100 10 100 2 Starmon-5 Superconducting 5 372
2025/02, 2026/09 [6][7] IBM x 100 10 100 9 ibm_marrakesh (Heron r2) Superconducting 156 2 400 195 000 300000
2025/02, 2026/09 [6][7] IBM x 100 10 100 9 ibm_fez (Heron r2) Superconducting 156 2 400 195 000 300000
2025/02 [6] IBM x 100 10 100 9 ibm_torino (Heron r1) Superconducting 156 2 400 210 000
2025/02 [6] IBM x 100 10 100 7 ibm_sherbrooke (Eagle r3) Superconducting 127 2 400 30 000
2025/02 [6] IBM x 100 10 100 7 ibm_kyiv (Eagle r3) Superconducting 127 2 400 30 000
2025/02 [6] IBM x 100 10 100 7 ibm_brisbane (Eagle r3) Superconducting 127 2 400 180 000
2025/02 [6] IBM x 100 10 100 7 ibm_quebec (Eagle r3) Superconducting 127 2 400 32 000
2025/02 [6] IBM x 100 10 100 7 ibm_kawasaki (Eagle r3) Superconducting 127 2 400 29 000
2025/02 [6] IBM x 100 10 100 7 ibm_brussels (Eagle r3) Superconducting 127 2 400 220 000
2025/02 [6] IBM x 100 10 100 7 ibm_rensselaer (Eagle r3) Superconducting 127 2 400 32 000
2025/02 [6] IBM x 100 10 100 7 ibm_strasbourg (Eagle r3) Superconducting 127 2 400 220 000
2024/08 [8] IQM x x 100 10 100 5 Garnet Superconducting 20 2 600
2026/09 [7] IBM x 100 10 100 ibm_phoenix (Nighthawk r2) Superconducting 120 2000000
2026/09 [7] IBM x 100 10 100 ibm_kingston (Heron r2) Superconducting 156 340000
2026/09 [7] IBM x 100 10 100 ibm_berlin (Nighthawk r1) Superconducting 120 25000
2026/09 [7] IBM x 100 10 100 ibm_aachen (Heron r3) Superconducting 156 330000
2026/09 [7] IBM x 100 10 100 ibm_miami (Nighthawk r1) Superconducting 120 24000
2026/09 [7] IBM x 100 10 100 ibm_boston (Heron r3) Superconducting 156 340000
2026/09 [7] IBM x 100 10 100 ibm_pittsburgh (Heron r3) Superconducting 156 330000

Motivation

The Circuit Layer Operations Per Second (\(\mathrm{CLOPS_v}\)) [3] was introduced by IBM in 2021. CLOPS is designed to simultaneously capture three critical aspects of quantum computation: quality, speed, and circuit scale. Specifically, the \(\mathrm{CLOPS_v}\) quantifies the number of Quantum Volume (QV) circuits [1] that a quantum processor can execute reliably per unit of time.

\(\mathrm{CLOPS_{h.v0}}\) is the variation of the \(\mathrm{CLOPS_v}\) protocol presented in 2023 and relies on the EPLG protocol rather that the Quantum Volume.

A third version called \(\mathrm{CLOPS_{h}}\) has been formalized in 2026 in [9].

Protocol

The initial protocol detailed in [3] is based on the Quantum Volume protocol and defines \(M\) template circuits. Each template circuit is composed of \(K\) different real quantum circuits with a set of parameter \(\vec{\theta}_i\) used to initialize each \(SU(4)\) gate. The permutation layers \(\pi_i\) are the same for all the circuits generated from a single template (see Fig. 1).

Bars and Stripes data set with segmentation of images that are in/outside the set.

The intial protocol introduced in [3] defines the figure of merit \(\mathrm{CLOPS_v}\) as:

\[\mathrm{CLOPS_v} = \frac{M \times K \times S \times D}{\mathrm{time\_taken}}\]

where \(M\) is the number of templates, \(K\) is the number of simulations done for each template with different parameters for the layers \(SU(4)\), \(S\) is the number of shots and \(D=\log_2 QV\) layers. The initial protocol sets \(M=100\) and \(K=10\), which leads to the execution of \(1000\) different quantum circuits. Each circuit is run with 100 shots. The \(\mathrm{time\_taken}\) accounts for the total execution time including data transfer to the QC, transpilation of high-level quantum gates to low-level control pulses, qubit measurement and reset time as well as inter-circuit delays.

Assumptions

Limitations

Update of the CLOPS protocol ( \(\mathrm{CLOPS_{h.v0}}\) ) - 2023/11

An update concerning the measurement of CLOPS values has been proposed in [2] with a new protocol measuring \(\mathrm{CLOPS_{h.v0}}\) (\(\mathrm{h}\) meaning hardware). This new protocol changes how layers are considered. In the initial protocol, a layer was defined by a random permutation of all the qubits followed by two-qubit gates (from the \(SU(4)\) group) across all the pairs of qubits. This initial protocol was favorable for fully connected topologies, which could more easily implement the random permutation than for topologies with restricted connectivity. The new protocol \(\mathrm{CLOPS_h}\) now relies on the EPLG protocol instead of the quantum volume. It splits sequences of gates that cannot be run in parallel into sublayers, artificially inflating the overall number of layers, which tends to increase the value of \(\mathrm{CLOPS_h}\) compared to \(\mathrm{CLOPS_v}\) even without any hardware improvement. It has the effect of inflating the CLOPS score for quantum computers that are sparsely connected.

Update of the CLOPS protocol (\(\mathrm{CLOPS_h}\)) - 2026/08

Another update concerning the measurement of CLOPS values has been formally presented in [9].

Implementations

An implementation of the \(\mathrm{CLOPS_v}\) based on the quantum volume developed by IQM is available here.
The \(\mathrm{CLOPS_{h.v0}}\) metric relies on the EPLG protocol. At the time of writing, an implementation of the complete \(\mathrm{CLOPS_h}\) protocol was not found.

References

  1. [1]A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, “Validating quantum computers using randomized model circuits,” Physical Review A, vol. 100, no. 3, p. 032328, 2019.
  2. [2]A. Wack and D. McKay, “Updating how we measure quantum quality and speed.” November 2023. Available at: https://www.ibm.com/quantum/blog/quantum-metric-layer-fidelity. [Accessed: February 26, 2025]
  3. [3]A. Wack et al., “Quality, Speed, and Scale: three key attributes to measure the performance of near-term quantum computers.” 2021. Available at: https://arxiv.org/abs/2110.14108
  4. [4]Rigetti, “Rigetti Computing Announces Commercial Availability of 80-Qubit Aspen-M System and Results of CLOPS Speed Tests.” 2022. Available at: https://investors.rigetti.com/node/7066/pdf. [Accessed: February 15, 2022]
  5. [5]W. van der Schoot, R. Wezeman, P. T. Eendebak, N. M. P. Neumann, and F. Phillipson, “Evaluating three levels of quantum metrics on quantum-inspire hardware,” arXiv preprint arXiv:2310.01120, 2023.
  6. [6]I. B. M. quantum, “Quantum processing units.” February 2025. Available at: https://quantum.ibm.com/services/resources. [Accessed: February 26, 2025]
  7. [7]I. B. M. quantum, “Compute resources.” 2026. Available at: https://eu-de.quantum.cloud.ibm.com/computers. [Accessed: September 19, 2026]
  8. [8]L. Abdurakhimov et al., “Technology and Performance Benchmarks of IQM’s 20-Qubit Quantum Computer,” arXiv preprint arXiv:2408.12433, 2024.
  9. [9]A. Wack, “CLOPS: Benchmarking System Speed at Utility Scale,” arXiv preprint arXiv:2608.18044, 2026.