QRosetta

Overview

You provide one circuit, written in OpenQASM 2.0. QRosetta sends that circuit to a set of simulator backends, currently including Qiskit (via Aer), Cirq, qsim, qibo, PennyLane, Qulacs, QuEST, quimb, myQLM, PyQuil, Qrisp, cuQuantum, and TorchQuantum. Several of these are accessed through pytket. Every backend receives the circuit in the same request format and returns its result in the same response format, so the results can be compared directly.

QRosetta then compares the backends pairwise. For each pair, it checks whether their outputs agree and, if they don't, whether a known cause explains the difference. See Agreement classification for how that works.

QRosetta compares outputs against each other; it has no independent way to check whether any of them is mathematically correct. If two backends disagree and you need to determine which one (if either) is right, that requires a different kind of analysis than QRosetta performs.

Quickstart

1. Enter an OpenQASM 2.0 circuit, or generate a preset (GHZ, Bell, QFT, Grover, Bernstein–Vazirani, VQE, random) or pull one from a benchmark suite.
2. Open the Config drawer to choose which backends to run, the timeout, optimization level, sampling seed, and (optionally) agreement thresholds.
3. Pick a run mode, statevector or measured, and run.
4. Read the report tabs: Summary, Performance, Resources, Divergence, Agreement, Artifacts, and Exports.

Circuit sources

A circuit can come from any of four places:

• Manual entry. Paste or type OpenQASM 2.0 directly into the editor.
• Presets. The circuit generator builds a preset family (GHZ, Bell, QFT, Bernstein–Vazirani, Grover, and others) at a qubit count you choose.
• QASMBench. A bundled suite of fixed-size circuits, selected by name.
• MQT Bench (Munich Quantum Toolkit). A bundled suite of algorithm-level circuits, selected by algorithm and qubit count.

The Benchmarks drawer covers the last two: pick a suite, then either load one circuit into the editor, add it to the batch queue, or use the bulk-select list (with a name filter and a qubit-count cap) to add many circuits to the batch queue at once.

Run modes

Statevector runs are limited to smaller qubit counts than measured runs, because a full statevector grows as 2ⁿ. When a circuit exceeds a backend’s limits, that pair is reported as not comparable rather than guessed at.

Batch runs and experiments

The Batch panel queues several circuits and runs them together as a single job, in either statevector or measured mode. Circuits can be added manually, from the generator, or from the Benchmarks drawer, including a whole suite at once capped by qubit count. A progress indicator shows which queued circuits are done, running, or still queued. The queue itself can be exported to a JSON file and re-imported later, for a repeatable circuit set.

Freezing a batch as a reproducible experiment

Checking "Freeze as reproducible experiment" inside the Batch panel runs the same queued circuits over one or more preprocessing axes, with each backend's own preprocessing pass on, off, or both, and packages the result into a single downloadable ZIP bundle containing:

• The exact input QASM for every circuit, frozen at run time.
• A conformance matrix recording whether each circuit ran successfully on each backend. Every error is recorded with an untriaged marker, and gets classified afterwards as an adapter issue, an SDK issue, expected/by-design behavior, or a resource limit. This classification step comes before any of it is treated as a finding.
• An agreement-prevalence table summarizing how often backend pairs agreed, and by which registered transform.
• A manifest recording the git commit, executed SDK versions, and the seed(s) used, so the whole sweep can be reproduced or cited.

Agreement classification

Two correct simulators can produce different-looking outputs for benign reasons. The two most common causes are a different qubit-ordering convention (endianness) and an unobservable global phase. For each backend pair, QRosetta checks a fixed set of known transforms, in order from smallest to largest effect, and reports the smallest one that makes the two outputs match, along with that transform's numerical effect.

Statevector labels

Measured labels

The shot-noise-aware statistical test

In measured mode a fixed distance threshold is only a heuristic. With few shots, two identical distributions can still look far apart purely from sampling. Alongside the descriptive JS/Hellinger label, QRosetta therefore reports a shot-noise-aware test:

• A chi-square and a G-test of homogeneity between the two count distributions;
• Benjamini–Hochberg (FDR) correction across the family of pairwise comparisons;
• Effect size (total variation, Cramér’s V) and a prospective power estimate.

This produces a verdict of consistent, distinguishable, inconclusive_low_shots, or ambiguous. The verdict is reported alongside the descriptive label, and it answers a different question: whether the observed difference is statistically significant, something a fixed distance threshold cannot tell you on its own.

Reproducibility

Every measured run is reproducible by construction:

• Every measured run has a recorded seed. Set a sampling seed in the Config drawer for an exact replay, or leave the field empty and QRosetta generates one for you and records it in the report. Statevector mode is exact and needs no seed.
• Executed SDK versions are captured. Each backend reports the exact version of the SDK it ran, and this is written into the report's provenance section.
• Provenance is exportable. Reports can be exported as an RO-Crate that conforms to the Process Run Crate profile, packaging inputs, per-backend results, agreement evidence, and provenance as a FAIR, citable bundle.

Timing and memory methodology

• Uniform warm-up. Every backend runs once, and that run is discarded, immediately before the timed measurement. This absorbs import time, JIT compilation, and any one-time internal caching. The same warm-up applies identically to every backend, so the reported timings are comparable.
• Process peak RSS. Resources are ranked by the kernel-reported (resource.getrusage ru_maxrss) peak memory of the whole isolated run, including compilation and the warm-up. Each backend's own memory-delta and simulation-window-peak figures are also reported for continuity, but are not the ranking basis.
• CPU time. User+system CPU time for the whole isolated run is reported alongside wall-clock time.
• Gate count and depth. Derived from each backend's own transpiled QASM, using a greedy-layering depth definition.

Reading the results

A batch or experiment run shows a reduced tab set: a suite summary and the Exports tab. A batch covers many circuits at once, so the full per-circuit breakdown above doesn't apply to it.

Accounts, history & autosave

An account is optional. Signing in (or registering with an email and password, via the Login / Register button) enables automatic run history: every completed run is saved to your account automatically, with no separate "save" step. Open Your Run History to browse past runs, rename them, download their full reports, or delete them (singly or in bulk). Without an account, results exist only in your browser for that session.

Separately, and independent of an account, the Config drawer offers autosave to a local folder: each completed run is written as a JSON file directly to a folder on your computer, with nothing uploaded anywhere. This needs the File System Access API (Chromium-based browsers such as Chrome and Edge) and degrades gracefully elsewhere.

Limitations

• Agreement between backends is a different thing from correctness. Backends agreeing does not prove they are all right, and a backend differing from the others does not prove it is wrong. QRosetta has no independent oracle to check against.
• Measured-mode thresholds are heuristic. The default JS/Hellinger cut-offs are configurable and depend on the shot count. The statistical test described above is the more rigorous companion to these thresholds.
• Permutation search is bounded. Exhaustive qubit-permutation matching is capped at 6 qubits. Larger cases report that the cap was hit rather than attempting an exhaustive search.
• A "residual disagreement" label is a starting point for investigation. It means that none of the registered conventions explained the difference. The underlying cause could be an unregistered convention, an environment limit, or a genuine defect in a backend or its adapter, and figuring out which one requires looking at the specific case.

Glossary

Agreement classes

See the label tables in Agreement classification above.

Statistical verdicts (measured mode)

FAQ

Why would two correct simulators disagree?

Usually because of a convention difference rather than an error, such as a different qubit-ordering (endianness) or an unobservable global phase. The agreement classification above exists to name these specific causes, so that a benign convention difference is identified rather than mistaken for a defect.

Is a "distance exceeds threshold" pair a bug?

It means the difference survived every registered transform, and in measured mode it may also be statistically significant. That result is worth investigating further. The cause could be an unregistered convention, a resource limit, or a genuine defect in a backend or its adapter. QRosetta gives you the evidence needed to investigate; determining the actual cause requires looking at that specific case.

Does QRosetta modify a backend's output?

No. Native outputs are stored exactly as each backend returned them. Bit-reversal, phase alignment, and permutation matching are analysis steps that run separately and get reported with their numerical effect, alongside the untouched native result.