Algorithmic Qubit benchmark
List of acronyms:
AC: Advanced Compilation method
CE: Constructor Evaluation (checked if the evaluation is done by the chip manufacturer)
COI: Conflict of Interest
EM: Error mitigation
SP: Scientific paper (checked if a scientific paper explain the results)
Ref | QPU Company | Year | COI risk | CE | SP | AC | EM | QPU | Technology | Chip qubits | #AQ | Comment |
---|---|---|---|---|---|---|---|---|---|---|---|---|
[1] | IonQ | 2023/03 | x | x | x | Aria | Trapped-Ion | 25 | 20 | |||
[1] | Quantinuum | 2023/03 | x | ? | ? | H1 model | Trapped-Ion | 20 | 12 | evaluation led by IonQ | ||
[1] | IBM | 2023/03 | x | ? | ? | Guadalupe | Superconducting | 16 | 6 | evaluation led by IonQ | ||
[1] | Rigetti | 2023/03 | x | ? | ? | Aspen-M1 | Superconducting | 80 | 5 | evaluation led by IonQ | ||
[2] | IonQ | 2023/09 | x | x | x | x | Forte | Trapped-Ion | 30 | 29 | ||
[3] | Quantinuum | 2024/03 | x | x | H2-1 | Trapped-Ion | 32 | 26 | ||||
[3] | IonQ | 2024/03 | x | x | Forte | Trapped-Ion | 30 | 9 | evaluation led by Quantinuum | |||
[3] | Quantinuum | 2024/03 | x | x | x | H2-1 | Trapped-Ion | 32 | 32 | |||
[3] | IonQ | 2024/03 | x | x | x | Forte | Trapped-Ion | 30 | 29 | evaluation led by Quantinuum | ||
[4] | IonQ | 2024/12 | x | x | x | Forte Enterprise | Trapped-Ion | 36 | 36 |
Algorithmic Qubit protocol
The Algorithmic Qubit (AQ) benchmark, introduced by IonQ in 2020 [5] is derived from the Volumetric benchmarking protocol [6]. A detailed description of the AQ protocol is available in an associated GitHub repository [7]. The protocol consists of sampling the output distribution of a quantum circuit and comparing this output to the ideal output distribution by computing the classical fidelity.
For validating a n AQ, several algorithms are used to generate the following quantum circuits using n qubits:
- 3 circuits based on the Quantum Fourier Transform
- 3 circuits based on the Quantum Phase Estimation
- 3 circuits based on the Amplitude Estimation
- 1 circuit based on the Monte Carlo Sampling algorithm
- 3 circuits based on a VQE simulation
- 1 circuit based on a Hamiltonian simulation
Each circuit is then compiled to a fixed gate set consisting of CX, Rx, Ry, and Rz using Qiskit compiler 0.34.2, with compilation seed set to 0. The depth and width of the compiled circuit are recorded as wd and wc (these quantities are used to build the volumetric plot).
It is important to emphasize that this first compilation step is performed solely to determine the quantities wd and wc. Subsequently, the circuit may be further compiled and optimized before execution on the quantum device. The only restriction is that the final circuit run on the quantum computer must implement the same unitary transformation as the initial circuit. Error mitigation techniques are permitted, provided that their application is clearly reported. Each circuit is sampled nshot times. For each circuit, the ideal sampling probability is denoted by ˜pc and the experimentally obtained distribution by pc. The classical fidelity F(pc,˜pc) is computed and the statistical error ϵc is extracted:
The number of algorithmic qubits associated with a quantum computer is defined as the largest size n such that all generated circuits satisfying wc≤n and depth √wd≤n achieve a fidelity beyond the validity threshold F(pc,˜pc)−ϵc>0.37.
Controversy on the algorithmic qubit
This protocol has been criticized for several reasons discussed in [7]. The authors highlight the problematic role of error mitigation techniques, which can artificially enhance the apparent performance of quantum computers and reduce the observed performance gap between different devices. They demonstrate that certain error mitigation methods, particularly those not expected to scale favorably with system size, can lead to misleading results. Additionally, they point out concerns regarding the use of Qiskit compiler version 0.34.2 for calculating the number of CX gates and hence, depth of the circuit. As this compilation method is used to compute the values used in the heatmaps wd and wc, a suboptimal compilation could artificially inflate a device’s reported performance. In particular, the authors show that alternative compilers can significantly reduce the overall number of gates (especially for the largest circuits).
Another limitation of the protocol is the restricted number of different circuits used during the benchmark which could affect the robustness of the results.
References
- [1]I. Q. staff, “Algorithmic qubits: a better single-number metric.” 2023 [Online]. Available at: https://ionq.com/resources/algorithmic-qubits-a-better-single-number-metric. [Accessed: 26-Feb-2025]
- [2]J.-S. Chen et al., “Benchmarking a trapped-ion quantum computer with 30 qubits,” Quantum, vol. 8, p. 1516, 2024.
- [3]Quantinuum, “Debunking algorithmic qubits.” 2024 [Online]. Available at: https://www.quantinuum.com/blog/debunking-algorithmic-qubits. [Accessed: 26-Feb-2025]
- [4]IonQ, “IonQ Unveils Its First Quantum Computer in Europe, Online Now at a Record #AQ36.” 2024 [Online]. Available at: https://ionq.com/news/ionq-unveils-its-first-quantum-computer-in-europe-online-now-at-a-record?utm_source=linkedin&utm_medium=social&utm_campaign=QuantumBasel&utm_content=press-release&utm_term=45627. [Accessed: 26-Feb-2025]
- [5]IonQ, “Quantum Benchmarking. Understanding Algorithmic Qubits.” 2020 [Online]. Available at: https://ionq.com/algorithmic-qubits. [Accessed: 26-Feb-2025]
- [6]T. Lubinski et al., “Application-oriented performance benchmarks for quantum computing,” IEEE Transactions on Quantum Engineering, vol. 4, pp. 1–32, 2023.
- [7]IonQ, “Rules for AQ v1.0.” 2022 [Online]. Available at: https://github.com/ionq/QC-App-Oriented-Benchmarks/blob/master/_doc/AQ.md. [Accessed: 26-Feb-2025]