Processing math: 100%

Algorithmic Qubit benchmark

List of acronyms:
AC: Advanced Compilation method
CE: Constructor Evaluation (checked if the evaluation is done by the chip manufacturer)
COI: Conflict of Interest
EM: Error mitigation
SP: Scientific paper (checked if a scientific paper explain the results)

Ref QPU Company Year COI risk CE SP AC EM QPU Technology Chip qubits #AQ Comment
[1] IonQ 2023/03 x x x Aria Trapped-Ion 25 20
[1] Quantinuum 2023/03 x ? ? H1 model Trapped-Ion 20 12 evaluation led by IonQ
[1] IBM 2023/03 x ? ? Guadalupe Superconducting 16 6 evaluation led by IonQ
[1] Rigetti 2023/03 x ? ? Aspen-M1 Superconducting 80 5 evaluation led by IonQ
[2] IonQ 2023/09 x x x x Forte Trapped-Ion 30 29
[3] Quantinuum 2024/03 x x H2-1 Trapped-Ion 32 26
[3] IonQ 2024/03 x x Forte Trapped-Ion 30 9 evaluation led by Quantinuum
[3] Quantinuum 2024/03 x x x H2-1 Trapped-Ion 32 32
[3] IonQ 2024/03 x x x Forte Trapped-Ion 30 29 evaluation led by Quantinuum
[4] IonQ 2024/12 x x x Forte Enterprise Trapped-Ion 36 36

Algorithmic Qubit protocol

The Algorithmic Qubit (AQ) benchmark, introduced by IonQ in 2020 [5] is derived from the Volumetric benchmarking protocol [6]. A detailed description of the AQ protocol is available in an associated GitHub repository [7]. The protocol consists of sampling the output distribution of a quantum circuit and comparing this output to the ideal output distribution by computing the classical fidelity.

For validating a n AQ, several algorithms are used to generate the following quantum circuits using n qubits:

Each circuit is then compiled to a fixed gate set consisting of CX, Rx, Ry, and Rz using Qiskit compiler 0.34.2, with compilation seed set to 0. The depth and width of the compiled circuit are recorded as wd and wc (these quantities are used to build the volumetric plot).

It is important to emphasize that this first compilation step is performed solely to determine the quantities wd and wc. Subsequently, the circuit may be further compiled and optimized before execution on the quantum device. The only restriction is that the final circuit run on the quantum computer must implement the same unitary transformation as the initial circuit. Error mitigation techniques are permitted, provided that their application is clearly reported. Each circuit is sampled nshot times. For each circuit, the ideal sampling probability is denoted by ˜pc and the experimentally obtained distribution by pc. The classical fidelity F(pc,˜pc) is computed and the statistical error ϵc is extracted:

ϵc=F(pc,˜pc)(1F(pc,˜pc))nshots

The number of algorithmic qubits associated with a quantum computer is defined as the largest size n such that all generated circuits satisfying wcn and depth wdn achieve a fidelity beyond the validity threshold F(pc,˜pc)ϵc>0.37.

Controversy on the algorithmic qubit

This protocol has been criticized for several reasons discussed in [7]. The authors highlight the problematic role of error mitigation techniques, which can artificially enhance the apparent performance of quantum computers and reduce the observed performance gap between different devices. They demonstrate that certain error mitigation methods, particularly those not expected to scale favorably with system size, can lead to misleading results. Additionally, they point out concerns regarding the use of Qiskit compiler version 0.34.2 for calculating the number of CX gates and hence, depth of the circuit. As this compilation method is used to compute the values used in the heatmaps wd and wc, a suboptimal compilation could artificially inflate a device’s reported performance. In particular, the authors show that alternative compilers can significantly reduce the overall number of gates (especially for the largest circuits).

Another limitation of the protocol is the restricted number of different circuits used during the benchmark which could affect the robustness of the results.

References

  1. [1]I. Q. staff, “Algorithmic qubits: a better single-number metric.” 2023 [Online]. Available at: https://ionq.com/resources/algorithmic-qubits-a-better-single-number-metric. [Accessed: 26-Feb-2025]
  2. [2]J.-S. Chen et al., “Benchmarking a trapped-ion quantum computer with 30 qubits,” Quantum, vol. 8, p. 1516, 2024.
  3. [3]Quantinuum, “Debunking algorithmic qubits.” 2024 [Online]. Available at: https://www.quantinuum.com/blog/debunking-algorithmic-qubits. [Accessed: 26-Feb-2025]
  4. [4]IonQ, “IonQ Unveils Its First Quantum Computer in Europe, Online Now at a Record #AQ36.” 2024 [Online]. Available at: https://ionq.com/news/ionq-unveils-its-first-quantum-computer-in-europe-online-now-at-a-record?utm_source=linkedin&utm_medium=social&utm_campaign=QuantumBasel&utm_content=press-release&utm_term=45627. [Accessed: 26-Feb-2025]
  5. [5]IonQ, “Quantum Benchmarking. Understanding Algorithmic Qubits.” 2020 [Online]. Available at: https://ionq.com/algorithmic-qubits. [Accessed: 26-Feb-2025]
  6. [6]T. Lubinski et al., “Application-oriented performance benchmarks for quantum computing,” IEEE Transactions on Quantum Engineering, vol. 4, pp. 1–32, 2023.
  7. [7]IonQ, “Rules for AQ v1.0.” 2022 [Online]. Available at: https://github.com/ionq/QC-App-Oriented-Benchmarks/blob/master/_doc/AQ.md. [Accessed: 26-Feb-2025]