Quantum Volume Benchmark

Benchmarking results

List of acronyms
CE: Constructor Evaluation (checked if the evaluation is done by the chip manufacturer)
SP: Scientific paper (checked if a scientific paper explain the results)
#Circ: number of circuits used for the evaluation

For clarity the Quantum Volume is expressed in a logarithmic basis.

Motivation

The main motivation for the Quantum Volume (QV) is to define a pragmatic way to evaluate and compare progress for near-term quantum computing. The QV is a single number figure of merit used to evaluate universal gate-based quantum computers. A quantum computer with a Quantum Volume (QV) of size \(n\) is able to reliably execute a quantum circuit with at most \(n\) qubits with maximum gate depth \(n\) [1].

Protocol

The Quantum Volume (QV) [24] [1] is a benchmarking protocol evaluating the ability of gate-based quantum computers to run quantum circuits reliably. This protocol gathers the maximum circuit width and depth that a quantum computer can successfully implement in a single metric number. A quantum computer has to successfully solve the Heavy Output Generation (HOG) problem [25] of size \(n\) to validate a quantum volume of size \(2^n\). Many criteria may impact the value of the quantum volume: gate fidelity, coherence time, chip topology, and the efficiency of the transpilation method.

Heavy Output Generation problem

The HOG problem is a sampling problem [25] considered hard for classical computers and constitutes a pass/fail test for quantum computers.

The problem is stated as:
Let \(Q\) be a random circuit drawn from a suitable ensemble acting on \(n\) qubits. The quantum state after executing the circuit is denoted \(\ket{\psi}\). Each possible output state \(x \in \{0, 1\}^n\) is measured with probability \(|\left<x|\psi\right>|^2\). The set of output states with a probability greater than the median constitutes the heavy set of outputs associated with the quantum circuit \(Q\).

Quantum Volume circuit

The following steps define how to generate quantum volume circuits:

• Choose a number \(n\) as the width of the circuit (i.e., the number of qubits in the circuit).
• Set the number of layers of the quantum circuit to \(n\).
• Generate a model circuit composed of \(n\) layers. Each layer comprises a random permutation (random relabelling) of qubits \(\pi\) and a random unitary sampled from \(SU(4)\). The permutation step is challenging for quantum chips with limited interconnections because it requires many swapping gates. The \(SU(4)\) gates can be decomposed into three CNOT gates and seven single-qubit rotation gates [26].

This circuit is then used as input for the sampling task associated with the Heavy Output Generation (HOG) problem. If the quantum computer samples the right distribution (simulated classically), it validates the corresponding quantum volume score of \(2^n\). The confidence interval for this evaluation is set to two-sigma (\(97.73 \%\)).

Assumptions

• The circuit compiler may use all the possible tricks to improve the mapping of the quantum circuit, which can lead to possibly high extra-processing time. In [4], the authors show that the compilation has a strong impact on small quantum volumes (\(< 10\)) as it can reduce the number of 2-qubit gates.
• The quantum computer should honestly attempt to solve the HOG problem and not choose an implementation far from the initial model of the circuit (i.e., the approximation error should be limited).

Limitations

• Quantum volume assesses only a limited subset of best-quality qubits within a quantum processor and does not provide a comprehensive measure of the overall fidelity or performance of quantum operations across the entire chip.
• Previous experiments [9] have shown that the Quantum Volume measured by quantum chip manufacturers is often hard to reproduce due to advanced optimization settings (compilation optimization and approximation of quantum gates) used to boost the performance of the quantum computer for passing this test.

Protocol Variations

In [4], the authors argue that the confidence interval built in the initial test [1] is more restrictive than necessary. They propose a new tighter confidence interval that still covers the initial requirement of \(97.73\%\) (using bootstrapping).

In [27], the authors extend the quantum volume protocol to verify the output sampling distribution for large experiments. This method uses parity tests to determine the heavy output probability distribution efficiently.

References

1. [1]A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, “Validating quantum computers using randomized model circuits,” Physical Review A, vol. 100, no. 3, p. 032328, 2019.
2. [2]A. Frisch et al., “Trapped-Ion Quantum Computing,” in Quantum Software, Springer Nature Switzerland, 2024, pp. 251–283 [Online]. Available at: http://dx.doi.org/10.1007/978-3-031-64136-7_10
3. [3]AQT, “IBEX Q1 Our quantum computer system that executes your quantum circuits via cloud access.” 2024 [Online]. Available at: https://www.aqt.eu/products/ibex-q1/
4. [4]C. H. Baldwin, K. Mayer, N. C. Brown, C. Ryan-Anderson, and D. Hayes, “Re-examining the quantum volume test: Ideal distributions, compiler optimizations, confidence intervals, and scalable resource estimations,” Quantum, vol. 6, p. 707, 2022.
5. [5]AQT, “MARMOT The first commercial 19-inch rack-mounted quantum computer.” 2024 [Online]. Available at: https://www.aqt.eu/products/marmot/
6. [6]N. Sundaresan, I. Lauer, E. Pritchett, E. Magesan, P. Jurcevic, and J. M. Gambetta, “Reducing unitary and spectator errors in cross resonance with optimized rotary echoes,” PRX Quantum, vol. 1, no. 2, p. 020318, 2020.
7. [7]P. Jurcevic et al., “Demonstration of quantum volume 64 on a superconducting quantum computing system,” Quantum Science and Technology, vol. 6, no. 2, p. 025020, 2021.
8. [8]IBM, “Processor types.” 2024 [Online]. Available at: https://docs.quantum.ibm.com/run/processor-types
9. [9]E. Pelofske, A. Bärtschi, and S. Eidenbenz, “Quantum volume in practice: What users can expect from nisq devices,” IEEE Transactions on Quantum Engineering, vol. 3, pp. 1–19, 2022.
10. [10]P. J. Karalekas, N. A. Tezak, E. C. Peterson, C. A. Ryan, M. P. Da Silva, and R. S. Smith, “A quantum-classical cloud platform optimized for variational hybrid algorithms,” Quantum Science and Technology, vol. 5, no. 2, p. 024003, 2020.
11. [11]J. M. Pino et al., “Demonstration of the trapped-ion quantum CCD computer architecture,” Nature, vol. 592, no. 7853, pp. 209–213, 2021.
12. [12]Quantinuum, “Quantuum Hardware Quantum Volume Data.” 2024 [Online]. Available at: https://github.com/CQCL/quantinuum-hardware-quantum-volume
13. [13]Honeywell, “Achieving quantum volume 128 on the Honeywell Quantum Computer.” 2020 [Online]. Available at: https://www.honeywell.com/us/en/news/2020/09/achieving-quantum-volume-128-on-the-honeywell-quantum-computer
14. [14]Quantinuum, “Demonstrating Benefits of Quantum Upgradable Design Strategy: System Model H1-2 Frist to Prove 2048 Quantum Volume.” 2021 [Online]. Available at: https://www.quantinuum.com/news/demonstrating-benefits-of-quantum-upgradable-design-strategy-system-model-h1-2-first-to-prove-2-048-quantum-volume
15. [15]Quantinuum, “Quantinuum Announces Quantum Volume 4096 Achievement.” 2022 [Online]. Available at: https://www.quantinuum.com/news/quantinuum-announces-quantum-volume-4096-achievement
16. [16]Quantinuum, “Quantinuum Sets New Record with Highest Ever Quantum Volume.” 2022 [Online]. Available at: https://www.quantinuum.com/news/quantinuum-sets-new-record-with-highest-ever-quantum-volume
17. [17]Quantinuum, “Quantum Volume reaches 5 digits for the first time: 5 perspectives on what it means for quantum computing.” 2023 [Online]. Available at: https://www.quantinuum.com/news/quantum-volume-reaches-5-digits-for-the-first-time-5-perspectives-on-what-it-means-for-quantum-computing
18. [18]S. A. Moses et al., “A race-track trapped-ion quantum processor,” Physical Review X, vol. 13, no. 4, p. 041052, 2023.
19. [19]Quantinuum, “System Model H2.” 2024 [Online]. Available at: https://www.quantinuum.com/products-solutions/quantinuum-systems/system-model-h2#h2-resources
20. [20]Quantinuum, “Quantinuum Dominates the Quantum Landscape: New World-Record in Quantum Volume.” 2025 [Online]. Available at: https://www.quantinuum.com/blog/quantum-volume-milestone
21. [21]L. Abdurakhimov et al., “Technology and Performance Benchmarks of IQM’s 20-Qubit Quantum Computer,” arXiv preprint arXiv:2408.12433, 2024.
22. [22]W. van der Schoot, R. Wezeman, P. T. Eendebak, N. M. P. Neumann, and F. Phillipson, “Evaluating three levels of quantum metrics on quantum-inspire hardware,” arXiv preprint arXiv:2310.01120, 2023.
23. [23]T. Jaeger et al., “Modeling Quantum Volume Using Randomized Benchmarking of Room-Temperature NV Center Quantum Registers,” arXiv preprint arXiv:2412.12959, 2024.
24. [24]L. S. Bishop, S. Bravyi, A. Cross, J. M. Gambetta, and J. Smolin, “Quantum volume,” Quantum Volume. Technical Report, 2017.
25. [25]S. Aaronson and L. Chen, “Complexity-theoretic foundations of quantum supremacy experiments,” arXiv preprint arXiv:1612.05903, 2016.
26. [26]F. Vatan and C. Williams, “Optimal quantum circuits for general two-qubit gates,” Physical Review A—Atomic, Molecular, and Optical Physics, vol. 69, no. 3, p. 032315, 2004.
27. [27]R. Bistroń, M. Rudziński, R. Kukulski, and K. Życzkowski, “Benchmarking quantum devices beyond classical capabilities,” arXiv preprint arXiv:2502.02575, 2025.