QRosetta

Overview

You provide one circuit, written in OpenQASM 2.0. QRosetta sends that circuit to a set of simulator backends: Qiskit (via Aer), Cirq, qsim, qibo, PennyLane (lightning and default.qubit), Qulacs, QuEST, quimb, myQLM, PyQuil, Qrisp, cuQuantum and TorchQuantum. GET /runners lists the ones the server has enabled. Several runners parse the QASM with pytket's parser. The Qiskit runner is one of them: it parses with pytket and converts the result with tk_to_qiskit. The cuQuantum runner, also built on Qiskit Aer, parses with Qiskit's own QuantumCircuit.from_qasm_str, so the two can read the same QASM differently; the Qiskit result's preprocessing_applied records the pytket parse. Every backend receives the circuit in the same request format and returns its result in the same response format, so the results can be compared directly.

QRosetta then compares the backends pairwise. For each pair, it checks whether their outputs agree and, if they don't, whether a known cause explains the difference. See Agreement classification for how that works.

QRosetta compares outputs against each other; it has no independent way to check whether any of them is mathematically correct. When two backends disagree, the report does not say which one is wrong: check the circuit against an independent reference, such as an exact simulation you write yourself. Known backend issues lists the disagreements testers have traced to a specific backend.

Quickstart: your first run

The page opens with a three-qubit GHZ circuit in the editor, Statevector ticked and every backend the server offers selected.

1. Press Run Editor Code, at the bottom of the left column, where Run, Mode, Run Name and the settings chips always stay in view. The button label says what will run, for example "Run Editor Code · Statevector · 14 backends". While the run is in progress, a card per backend shows its state.
2. Read the Summary tab. Its first line says how many backends returned a result; the next line names any backend that differs from the others, for example "Qrisp differs from 13 of 13". A GHZ circuit should read "Every comparable pair of backends agrees". The Backends table gives one row per backend: its status, how many others it differs from, its bit order, precision, time and peak memory. Below it, Evidence lists the pairs grouped by verdict.
3. For a measured run, choose Sampler bias, 16 qubits in the Examples group of the Quantum Algorithms menu and press Load. This ticks Measured. Open the Seed chip above the run button and set a Sampling seed (for example 42) so a rerun draws the same shots, then press Run Editor Code. Expect the full-distribution test to call most pairs inconclusive (16 qubits have 65,536 outcomes, far more than 8,200 shots cover; the Summary gives the shots the test would need) and the single-bit marginal test to name Qrisp, whose sampler is biased on this circuit (see Known backend issues).
4. Open the other tabs when you need them: Agreement (a heatmap of every pair, the evidence per pair, and the raw matrices), Performance (times and memory), Circuits (what each backend executed), Exports (figures and tables you set up, the RO-Crate and the report files).

Inputs are checked before a run is submitted: a shot count, timeout, seed or threshold that is out of range is marked next to the field and the run is not sent. Requests the server rejects are shown with the server's message. If the page is reloaded while a run is in progress, the page offers to show that run's results when it loads again; the editor and the playlist are kept in this browser.

Circuit sources

Each panel on the left has one job: Generate Circuits chooses a ready-made circuit, the Editor holds the circuit you are working on, and the Playlist holds circuits to run together.

• Generate Circuits. The Quantum Algorithms menu holds every ready-made circuit in four groups: the generated algorithms (GHZ, Bell, QFT, Bernstein–Vazirani, Grover, VQE ansatz, random, bit-order probe), bundled Examples (GET /examples), MQT Bench algorithms and QASMBench circuits. Type in it to search all four. For an algorithm or an MQT Bench algorithm, Qubits sets the size (an MQT Bench size moves to the nearest one the suite has); examples and QASMBench circuits have a fixed size. The line under the menu describes the selection; for an example it names the mode it is meant for. Load puts the selection in the editor (asking first when that replaces your edits; loading an example ticks its mode); + Add adds it to the playlist.
• Editor. Paste or type OpenQASM 2.0; Import QASM reads a file into it and Export QASM saves it; + Add to Playlist adds the editor's circuit to the playlist under the name above the editor.
• Playlist. Quick Load fills it with a ready-made set (one of each algorithm, a GHZ scaling series, or the largest statevector circuits); Auto-Gen adds circuits of chosen algorithms at random qubit counts; Bench opens a list of MQT Bench or QASMBench circuits beside the button, with a name filter and a qubit cap, where you tick circuits and add them, or add every circuit the list shows (with no filter and no cap, the whole suite). Escape or a click elsewhere closes it.

In the run section at the bottom of the left column, Run switches between Editor code and Playlist; a warning appears there when a circuit has more qubits than a ticked mode allows. Loading a circuit into the editor selects the editor; adding to the playlist selects the playlist. The playlist can be exported to and imported from a JSON file.

Run modes

In statevector mode the API removes measurements at the end of the circuit before dispatch and lists that as an API: entry in preprocessing_applied. A circuit with a measurement followed by further operations, a reset, or an if is rejected with HTTP 400 in statevector mode, because it has no single output statevector; run it in measured mode. In measured mode these operations are passed to every backend; adapters that cannot simulate them return an error for that backend. Qiskit and cuQuantum (Aer) run mid-circuit measurement, reset and if; Qrisp runs mid-circuit measurement and if; Cirq and qsim run mid-circuit measurement; the other backends refuse all three.

Statevector runs are limited to 18 qubits and measured runs to 24 (the server reports its limits at GET /config), because a full statevector grows as 2ⁿ. A compile-only run builds no statevector and has its own limit of 24 qubits. Each backend's run ends with one of four dispatch statuses: ok, error (the backend rejected the circuit or crashed), timeout (the run exceeded the timeout, or never left the runner's queue; see Resources, queueing and timeouts) or offline (the runner could not be reached). A failed run's timing and memory fields are null, not 0. Any pair involving a failed run is reported as not comparable.

Each result can also report device (CPU or GPU), simulator_backend (the library that executed the circuit) and simulation_method. The Summary's per-backend details and the Performance tables show them as tags. Qiskit runs on qiskit-aer's AerSimulator with method="statevector". The cuQuantum runner is qiskit-aer-gpu 0.15.1 with cuStateVec enabled, running on an NVIDIA GPU; its results also carry gpu_name and custatevec_enabled. On a server without an NVIDIA GPU it is listed as unavailable in the Backends chip's list, with the reason the server gives, and it is not dispatched; it never falls back to the CPU. Its kernels run on the GPU, while its host-side work (parsing, launching kernels, copying results back, sampling) runs under the same CPU quota as the other runners (see Resources, queueing and timeouts). A time comparison between cuQuantum and a CPU backend therefore compares different hardware.

Each backend reports the amplitude precision it simulated in. Most use complex128; qsim, Qrisp, TorchQuantum and Cirq's measured path use complex64, and cuQuantum reports the precision its simulator was configured with. The report tabs show a precision badge per backend; a complex64 backend's fidelity against a complex128 one carries single-precision rounding error.

Circuit preprocessing

Changes to the circuit happen in two places, and both are recorded in each backend's preprocessing_applied list.

• API-side (entries starting API:, tagged API in the UI). The API prepares one circuit per dispatch group. With Apply SDK Preprocessing on:
  • Statevector mode: terminal measurements are removed, several quantum registers are merged into one in declaration order, and the circuit is parsed and re-emitted with pytket with barriers removed.
  • Measured mode, both groups (measured_native, backends that sample with their own sampler, and measured_sampled, backends that sample a statevector): register merge, then the pytket round-trip with barriers removed.
  The pytket round-trip inlines custom gate definitions, rewrites gates into pytket's QASM gate set (for example u becomes u3 and cp becomes cu1, with angles as multiples of π) and expands register broadcasts. tket sets a parameter within 1e-11 half-turns of a multiple of π/4 to that multiple; when that happens the API: entries list each moved parameter (see Known backend issues). In measured mode the native group also receives a measure-all when the circuit has no measurements; this happens with preprocessing off too. With preprocessing off there is no register merge and no pytket round-trip in any group; statevector mode still removes terminal measurements. The report's dispatched_circuits lists which runners were in which group.
• Runner-side. Each backend's own passes (gate rebasing, barrier stripping, the backend's default compilation pass at every optimization level). Passes the adapter needs regardless of the setting say so in their text. The qibo and quimb parsers lack some qelib1.inc gates (ch, cswap, c3x, c4x, rccx, rc3x, c3sqrtx, u0 for qibo; u, p, ch, csx, cp, cu, rccx, c3x, c3sqrtx, rc3x, c4x for quimb); with preprocessing on, their adapters insert Qiskit 2.5.2's qelib1.inc definitions of the ones a circuit uses (for quimb with the builtin U written as u3, which qelib1.inc defines as U) and list them. With preprocessing off those gates are rejected.

The RO-Crate stores the original circuit (circuit.qasm), the circuit the API dispatched to each runner (dispatched_circuits/<runner>.qasm, one file per runner; <circuit>/<runner>.qasm in a batch crate) and the circuit each backend executed (transpiled_circuits/).

Run options

These controls sit in the settings chips above the run button (Backends, Execution, SDK Preprocessing, Compile-only services, Agreement Thresholds, Seed); each chip shows its current value, is highlighted when that differs from the default, and opens its controls beside it. They apply to single runs, runs of the playlist and experiments alike. Each has an API field of the same meaning, listed in the Swagger UI at /docs.

• Timing repeats (timing_repeats, 1–10, default 1). All repetitions run in one process and give one result per backend; to compare separate runs of a backend with each other, use Multi-run mode (below). The number of timed runs per backend after the one discarded warm-up run. With more than one, each result carries compilation_time_samples and simulation_time_samples, their mean, median and standard deviation, timing_samples (the number of timed runs) and timing_statistic: "median"; the Performance tab shows the median ± the sample standard deviation with n. All repeats and the warm-up share one timeout.
• Exclusive timing run (exclusive, default off). Every backend runs in its own phase, one after another, and the job runs only while no other run is in flight on this server: it waits for running jobs, and jobs submitted meanwhile wait for it. Results record exclusive and exclusive_wait_sec; every result records gate_wait_sec, so a job held back by another user's exclusive run shows how long it waited, and its runners show as queued meanwhile. When an exclusive run ends, the jobs it held start together and overlap each other (server_overlapping_runs). It overrides runner_phases. It cannot hold back other programs on the host, and in Kubernetes mode it covers only this server's runner Jobs. Each client may have one unfinished exclusive job (another returns 429). An exclusive circuit that cannot start within the server's wait cap (exclusive.max_wait_sec in GET /config, default 600 s) fails with that reason; GET /config also shows whether an exclusive run is running or waiting. Operators can turn the option off (QROSETTA_EXCLUSIVE_ENABLED=false).
• Shots (n_shots, measured mode, default 8200): enough for the power model to reach 0.8 on circuits with up to 16 outcomes and 14 backends (8199 needed). Circuits with more outcomes need more; the Summary recommends a count when it can.
• Multi-run mode (self-comparison), under Backends. Runs one backend several times, each in its own process, at the same or different optimization levels, and compares the runs with each other like separate backends. In the API, a list in runner_config runs that backend once per element, for example {"qiskit": [0, 0, 1]}. Each run gets the key <runner>~opt<L>~run<N>, and the key is part of its seed derivation. A request may ask for at most 16 runs of one backend and 48 runs in total; more is rejected with HTTP 422.
• Custom ordering (phases) (runner_phases), under Backends. Assigns each backend a phase number: backends with the same number run at the same time, and phases run in ascending order. Without phases every backend runs at once. Phases change which backends compete for the host at the same moment, not the per-runner CPU quota.
• Bypass API Safety Limits (bypass_limits, local mode only). Skips the QASM size limit and the statement-count limit. The qubit limits, the expanded-gate and nested-gate-call limits (which protect the runners' memory), the syntax checks and the measurement checks still apply, and the server ignores the flag outside local mode. The report records it.
• Apply SDK Preprocessing: on or off (apply_preprocessing), or both, which runs an experiment (Several circuits, several runs, and experiments); see Circuit preprocessing.

Presets worth knowing

The Bit-order probe preset (at least 2 qubits) applies x to qubit 0 and a different ry angle to every other qubit, so the output is not symmetric under any qubit permutation. It shows each backend's declared bit order directly: an lsb backend and an msb backend agree on it only after the bit reversal their declarations imply.

Input limits

• At most 18 qubits in statevector mode, 24 in measured mode and 24 in a compile-only run, counted over every qreg declaration.
• The QASM text may hold at most 1,000,000 characters (HTTP 413 above that). This is separate from the request-body limit of 1,048,576 bytes (1 MiB) that applies to every request, including POST /export.
• At most 10,000 top-level operation statements (gates, measure, reset, barrier, if; max_qasm_gates in GET /config). The header, the include line, register declarations and gate definitions are not counted.
• At most 200,000 gate applications after expanding user gate definitions and register broadcasts (limits.max_expanded_gates), so a short circuit of nested gate definitions cannot expand to millions of gates, and at most 4,096 applications of user-defined gates made from inside gate definitions in that expansion (limits.max_nested_gate_calls; top-level calls are not counted): several parsers keep a copy of a nested definition for each application. Eleven levels of definitions that each call the one below twice (4,094 nested applications) peak at about 710 MiB in the PennyLane runners and twelve levels at about 1,090 MiB, while 8,192 top-level calls of one definition take about 340 MiB. The bundled benchmark circuits are exempt.
• A custom gate definition with a name pytket's QASM parser treats specially is rejected with HTTP 400, and the message names the gate. pytket turns some names (sleep, order2…order20, group2…group20) into barriers and replaces others with its own built-in operation (for example fsim, yyphase, iswap, eswap, phasediswap, xxphase3, tk2) whatever the definition says. Every pytket-parsed backend would then run a different circuit from the one written and agree with the others on it. Rename the gate.

Quirk visualizer

The editor's Visualize button converts the circuit in the editor to Quirk's JSON format (POST /visualize/quirk) and opens it in Craig Gidney's Quirk, an external web page, inside a frame. The conversion covers the gates Quirk has; a circuit with other gates shows an error instead.

Several circuits, several runs, and experiments

One press of the run button can start more than one job:

When there are two jobs, a row above the tabs switches between their results. A batch shares one master seed, and because a backend's runner_seed is derived from the master seed and its runner key only, each backend receives the same runner_seed in every task. Each task is named after its circuit (name in the request, else the preset or file name) and qubit count; a bundled benchmark circuit adds its suite, as in ghz (4q) [MQT Bench], so it is not confused with a preset of the same name.

Experiments: preprocessing on and off

With Both selected, the circuits run once with each backend's preprocessing on and once with it off, and the result is one ZIP bundle containing:

• The input QASM for every circuit, frozen at run time.
• A conformance matrix recording whether each circuit ran on each backend. Every error carries an untriaged marker for you to classify as an adapter issue, an SDK issue, by-design behaviour, or a resource limit. The software does not classify errors itself.
• An agreement-prevalence table: how many backend pairs fell in each class, by measured_assessment in measured mode and by label in statevector mode.
• A manifest recording the git commit, executed SDK versions, the seeds used and the effective configuration.

In the API the experiment is POST /experiments with a list of circuits and preprocessing_modes ([true], [false] or [true, false]); the bundle is GET /experiments/{job_id}/bundle. Nothing is stored on the server beyond the job's retention time (30 minutes), and experiments are not saved to the run history.

Agreement classification

Two simulators can produce different-looking outputs because of a different qubit-ordering convention (endianness) or an unobservable global phase. For each backend pair, QRosetta checks a fixed list of transforms in order and reports the first one under which the two outputs agree, with the metric before and after it.

A statevector pair agrees when its fidelity, computed on normalised vectors and stored unrounded, is at least 1 − fidelity_tolerance. The default tolerances are fidelity_tolerance = 1e-5, phase_tolerance = 1e-6 rad, norm_tolerance = 1e-4, js_threshold = 0.01 nats and hellinger_threshold = 0.1 (the server reports its defaults at GET /config). They can be changed in the Agreement Thresholds chip; the values sent are recorded in the report's run configuration and in every pair record, and the pair record's values are the ones the label was computed with. The phase tolerance of 1e-6 rad sits above single-precision rounding (about 1e-8 rad for complex64), so a complex64 backend is not reported as carrying a global phase because of rounding alone.

Statevector labels

A statevector pair is labelled in the basis its two declared qubit orderings imply (metrics.label_aligned_basis): the bit-reversed basis for an lsb backend against an msb backend, the direct basis otherwise. A one-qubit pair, or a pair where a backend declares no ordering, uses the direct basis (metrics.basis_source says which rule applied). The labels fall into three groups.

Measured labels

The measured labels are descriptive distance checks, and at small shot counts two samples of the same distribution routinely exceed them. Whether a measured pair disagrees is decided by its measured_assessment, derived from the shot-noise test:

For a report without measured_assessment, the UI reads metrics.statistical_test.verdict instead (consistent, inconclusive_low_shots, distinguishable, ambiguous). A measured pair whose observed shot totals differ from the requested n_shots records both in metrics.shot_total_mismatch.

Metric formulas

Statevectors

Each backend's statevector a is normalised first: â = a / ‖a‖₂. The norm ‖a‖₂ is recorded in metrics.norms; if |‖a‖₂ − 1| > norm_tolerance for either side, the pair is flagged unnormalized_input.

• Direct fidelity: F = |⟨â|b̂⟩|² = |Σₖ conj(âₖ) b̂ₖ|², clamped to [0, 1] and stored unrounded.
• Global phase offset: φ = arg⟨â|b̂⟩ in radians, in (−π, π]. A value with |φ| ≤ phase_tolerance is reported as 0. This is the overall phase between the two states; a relative phase between components of a superposition shows up as reduced fidelity instead. The JSON field is direct_relative_phase_rad and the matrix is relative_phase_matrix.
• Bit-reversed fidelity: F_rev = |⟨â|R b̂⟩|², where (R b)ₖ = b_rev(k) and rev(k) reverses the n-bit binary representation of the index k.
• Permutation fidelity: the largest |⟨â|P_π b̂⟩|² over every permutation π of the n bit positions other than the identity and the reversal; P_π reshapes the vector to n axes of size 2 and transposes them by π. Ties keep the lexicographically first permutation. The search runs for n ≤ 7 qubits; above that the pair records too_many_qubits.

Labels are assigned in this order, with ε = fidelity_tolerance and the label-aligned fidelity F_al (F in the direct basis, F_rev in the bit-reversed basis; recorded as metrics.label_aligned_fidelity): in the direct basis, agree_directly if F ≥ 1 − ε and |φ| ≤ phase_tolerance, and agree_up_to_global_phase if F ≥ 1 − ε; in the bit-reversed basis, agree_after_bit_reversal if F_rev ≥ 1 − ε. Otherwise, if the fidelity in the other basis is ≥ 1 − ε, residual_disagreement with unaligned_basis_match; else agree_after_qubit_permutation if the permutation fidelity is ≥ 1 − ε; else residual_disagreement. An agreeing pair with unnormalized_input is relabelled agree_after_normalization. The Summary quotes the fidelity under the transform the label names, with the direct fidelity beside it.

Counts

Each backend's counts c(x) are normalised by its own observed total: p(x) = c(x) / Σₓ c(x), over the union of outcomes of the two backends.

• Jensen–Shannon divergence: JS(p, q) = ½ Σ p ln(p/m) + ½ Σ q ln(q/m) with m = (p + q)/2, natural logarithm, so it is in nats and lies in [0, ln 2 ≈ 0.693].
• Hellinger distance: H(p, q) = (1/√2) ‖√p − √q‖₂ = √(1 − Σ √(p q)), in [0, 1].
• Total variation distance: TV(p, q) = ½ Σ |p − q|, in [0, 1]; recorded as statistical_test.effect_size.total_variation.

A measured pair is distance_within_threshold when JS ≤ js_threshold and H ≤ hellinger_threshold; the bit-reversed and permuted counts are tried next, as for statevectors.

Shot-noise tests

The two backends' counts form a 2 × K contingency table O over the K outcomes observed by either backend (outcomes with zero counts on both sides are dropped). The table is built in the label-aligned basis: when the two backends declare different bit orders (lsb against msb), the second backend's keys are bit-reversed first; otherwise the keys are compared directly. The basis comes from the declared orderings, not from which basis gives the smaller distance, and is recorded as statistical_test.basis. Outcomes whose expected count is below 5 are pooled into one bin, and more of the rarest outcomes are merged in until the pooled bin reaches an expected count of 5 or the table meets Cochran's rule (no expected count below 1, at most 20% below 5); pooled_outcomes records how many outcomes were merged. With row totals Rᵢ, column totals Cⱼ and N = Σ O, the expected counts are Eᵢⱼ = Rᵢ Cⱼ / N.

• Pearson chi-square: χ² = Σ (O − E)² / E, no continuity correction, df = K − 1.
• G-test: G = 2 Σ O ln(O / E) divided by Williams' correction q = 1 + (N Σ 1/Rᵢ − 1)(N Σ 1/Cⱼ − 1) / (6 N (rows − 1)(K − 1)), same df (g_test.statistic; the uncorrected value is statistic_uncorrected and q is williams_q). Without it the G-test rejects far too often on tables with many bins of small expected counts: in simulation with no real difference, 95% of 65,536-bin tables at 10⁶ shots were rejected at α = 0.05, 15% with the correction.
• Cramér's V: √(χ² / N).
• Multiple comparisons, over the m tested pairs of the run, separately for each test. Benjamini–Yekutieli (correction: "BY"): with p-values sorted p₍₁₎ ≤ … ≤ p₍ₘ₎ and c_m = Σᵢ₌₁..ₘ 1/i, p̃₍ₖ₎ = min over j ≥ k of min(1, m c_m p₍ⱼ₎ / j), recorded as p_value_by. A test is significant (significant_by) when its BY-adjusted p-value is below α = 0.05. BY controls the false discovery rate at q = α under any dependence between the tests, which matters here because pairs that share a backend share that backend's sample. Among the pairs it flags, up to a fraction q are expected to be flagged by chance, so a few flagged pairs in a run with many can be false discoveries, and one unusual sample of one backend can flag several of its pairs at once. The Benjamini–Hochberg (p_value_bh) and Bonferroni (min(1, m p), p_value_bonferroni) adjusted p-values are recorded for reference, and so are Holm's step-down adjusted p-values (p_value_holm, significant_holm), which control the family-wise error rate. A distinguishable pair carries significant_fwer: true when both tests also pass Holm. Under BY, genuine discoveries in other pairs lower the bar for every pair, so with one wrong backend among 14, a pair of two correct backends is flagged in about 8–9% of runs; the Summary marks a distinguishable pair that fails Holm as significant under BY only. A p-value that underflows to 0 is shown as <1e-300, and values below 1e-3 in scientific notation.
• Verdict: inconclusive_low_shots (low_expected_counts) if no pooling gives adequate expected counts; else distinguishable if both tests are significant, ambiguous if exactly one is, and, if neither is, consistent when the power at the observed shots, computed at α/(m·c_m), is at least 0.8 and inconclusive_low_shots (low_power) otherwise. resolvable_by_more_shots, beside inconclusive_reason, says whether more shots can turn an inconclusive pair into a verdict. It is null, and no shot count is recommended, when the sample does not show the outcome distribution: unseen_mass_good_turing, the share of shots in outcomes seen exactly once (the Good–Turing estimate of the probability of outcomes not yet seen), exceeds 0.5 for either backend (support_characterised: false). That happens when the circuit spreads its outcomes over far more basis states than the shots cover, for example a 22-qubit dense state at 8,200 shots; measuring only the qubits of interest gives the test a distribution it can resolve. When the unseen mass is between δ/2 and 0.5, the recommendation is given as a lower bound (recommendation_is_lower_bound): new outcomes appear with more shots and can raise it. For a 12-qubit QFT at 8,200 shots it recommended 84,838, and at 100,000 shots every pair was consistent.
• Power model: the effect size is Cohen's w = δtested, the smallest w any distribution pair at total variation δ = 0.05 can have on the table the test sees (w ≥ TV). Pooling rare outcomes into one bin can hide up to min(P, Q) of the total variation, where P and Q are the two backends' probabilities in the pooled bin, so δtested = δ − min(P, Q) (recorded as power.delta_tv_tested and power.pooled_probability). When δ_tested ≤ 0 a gap of δ could sit entirely inside the pooled bin, the power is α′ and the pair is inconclusive, however small its p-values. The chi-square statistic then follows a noncentral χ² with df = K − 1 and noncentrality λ = N w². power_to_detect_delta_at_observed_shots = 1 − F(χ²₁₋α′,df ; df, λ) with α′ = α/(m·c_m), the level BY applies to a single discovery among the m tested pairs of the run (recorded as power.alpha_used, power.m_pairs and power.c_m). It is the smallest level BY applies to any pair, so the power figure does not overstate the chance that the corrected test flags a gap of size δ. It is still the power of the chi-square test alone; the verdict needs both tests to be significant. The shots needed therefore grow with the number of outcomes (df) and with the number of backends (m); the report computes the power for each pair rather than assuming the default shot count is enough. recommended_shots_per_backend (also recorded as shots_per_backend_for_target_power) is ⌈N*/2⌉, where N* is the total giving power 0.8 at α/(m·c_m); when outcomes were pooled, it is the smallest count at which the observed proportions, scaled to that count, pool into a table with that power. recommended_exceeds_actual says whether that is more than the shots either backend used; the Summary recommends more shots only then. Because df uses the observed outcomes, the recommendation can be too small when the true support is larger.

Bit order per backend

Each result declares qubit_ordering: msb when qubit 0 is the most significant bit of the statevector index or counts bitstring, lsb when it is the least significant. The labels below are what each runner declares; tests/integration/test_runner_contract_asymmetric.py checks them against an asymmetric circuit on the live stack. Statevector and counts pairs are compared in the basis these declarations imply. A pair that matches only in the other basis (after bit reversal although both declare the same ordering, or directly although they differ) is a disagreement with unaligned_basis_match: either a label below is wrong or a gate defect mimics a reversal.

Counts keys

A counts key is a bitstring over every classical bit the circuit declares, including bits no measure statement writes; those read 0. Classical bits get one flat index in creg declaration order: with creg c[2]; creg d[3];, c[0], c[1] are bits 0 and 1 and d[0]…d[2] are bits 2–4. In an lsb key, classical bit 0 is the rightmost character; in an msb key it is the leftmost. A circuit without measurements, in measured mode, gets a measure-all appended for backends that sample natively (see Circuit preprocessing).

The shot-noise statistical test

In measured mode a fixed distance threshold is descriptive. With few shots, two samples of the same distribution can still differ noticeably. Alongside the JS/Hellinger label, QRosetta reports for every comparable pair:

• A Pearson chi-square test (no continuity correction) and a G-test of homogeneity between the two count vectors, in the basis the two backends' declared bit orders select, with rare outcomes pooled;
• Benjamini–Yekutieli (FDR under any dependence) adjusted p-values across all pairs of the run, which decide significance, with Benjamini–Hochberg and Bonferroni adjusted values recorded beside them;
• Effect size (total variation, Cramér’s V) and a power estimate at α/(m·c_m) that uses the observed degrees of freedom (power.df).

The verdict is consistent, distinguishable, inconclusive_low_shots or ambiguous, from the BY-adjusted p-values at α = 0.05; Metric formulas gives the statistics, the corrections and the power model. Each backend samples with its own derived seed, so two backends never draw the same shot sequence; shared_seed records whether a pair's runner seeds were equal.

Single-bit marginals. A circuit with many outcomes (256 or more) often gets inconclusive_low_shots at the default shots, because most outcomes are seen a few times. As a second, separate analysis every pair whose keys have two or more bits is also compared bit by bit: for each classical bit, the two backends' counts of that bit being 0 and 1 form a 2×2 table, tested with the same chi-square and G-test. All bit tests of all pairs form one family under Holm's correction at α = 0.05, so the chance of any false flag in a report is at most α however many backends really differ, and a bit differs when both tests are significant. A bit with an expected count below 5, or constant in both samples, is not tested. Equal distributions have equal marginals, so a differing bit shows that the two distributions differ; marginals that do not differ do not show that they agree. The full-distribution tests and the marginal tests are two families, each at α. Each pair records marginal_assessment (marginal_distinguishable, no_marginal_difference_detected, not_tested or not_applicable) and metrics.marginal_test (each bit's P(1) on both sides, raw and Holm-adjusted p-values); measured_assessment is not changed by it, so a pair can be consistent_within_shot_noise (no total-variation gap of 0.05 found) and still have a differing marginal. The Summary lists such pairs in their own block, the Agreement tab marks them "marginal differs", batch badges count them as an anomaly, and crates and experiment bundles count them by marginal_assessment. In a simulation with 14 backends over 16 independent bits at 8,200 shots, 1 of 40 runs had a false flag when all backends sampled the same distribution, 1 of 40 had one on a pair of correct backends when one backend was off by 0.05 in P(1) on four bits, and every pair with the wrong backend was flagged.

Reproducibility

• Seeds. Every measured run has a master seed: the one you set in the the Seed chip, or one the API generates and records at seed. Each backend receives runner_seed = the first 4 bytes of sha256("<master seed>:<runner key>"), read big-endian, top bit cleared; the runner key includes any ~opt<L>~run<N> suffix. The formula is recorded at seed_derivation. seed_honoured is false for a backend whose sampler ignores the seed (Braket, which is disabled on this server). In a batch every task uses the same master seed, so each backend receives the same runner_seed in every task.
• Executed SDK versions are captured by each backend and written into the provenance.
• Run configuration. Reports record the agreement thresholds sent, the timeout and its scope, the optimization levels, apply_preprocessing, the API-side preprocessing, the dispatched circuits, started_at / finished_at and the git commit the API image was built from (git_sha; a "-dirty" suffix marks uncommitted changes). The divergence report records analysis_version, the version of the classification and test rules. Runner image digests are recorded only when the deployment sets QROSETTA_IMAGE_DIGESTS; in local mode the report says "not recorded". Each result carries image_git_sha, the commit its runner or compiler image was built from, and runner_code_sha256 (for compilers compiler_code_sha256): sha256 over the bytes of the image's main.py followed by the bytes of every .py file of the installed qrosetta_commons package, in order of relative path. It identifies the adapter that produced the result; recompute it from a checkout with the same concatenation, not from main.py alone.
• Statevector hashes. A report stored or exported without statevectors (history, RO-Crates with include_statevectors false, experiment bundles) keeps statevector_sha256 and statevector_length for every backend. The hash is not stable across replays for every backend: Cirq, for example, can differ in the last bits of some amplitudes between two runs of the same circuit. Compare replayed statevectors by fidelity.
• RO-Crate export. A run exports as an RO-Crate conforming to the Process Run Crate profile, with the inputs, dispatched and executed circuits, per-backend results (counts, or statevector hashes unless statevectors are included), agreement evidence and provenance.

Replaying a run

1. Take the circuit from the crate's circuit.qasm (circuits/ for a batch) or the experiment bundle's inputs/*.qasm, which hold the text as submitted; for a bundle, check each file against the manifest's qasm_sha256. Submit it as literal QASM; do not regenerate a preset.
2. Send the effective values rather than the requested ones: the seed from results.seed (for an experiment, effective_seeds_per_axis); the thresholds from the report's agreement_thresholds, sent explicitly, because defaults can change between versions; an explicit runner list taken from raw_results[].simulator, because an empty list means every runner enabled at the time; and n_shots, apply_preprocessing, optimization_level, runner_config, runner_phases, timing_repeats, timeout_seconds and, when the run used compile-only services, compilers from the report's compilers_requested. The experiment manifest's effective_config records presets by name; replay them from the inputs/*.qasm files as literal QASM, as in step 1.
3. Compare in this order: runner_seed per backend; the dispatched and transpiled circuits, and compile_input.qasm and compiled_circuits/; counts, byte for byte; statevectors by fidelity (this needs the statevectors, so export with include_statevectors or use GET /jobs/{job_id}/report; from hashes alone only a bit-identical replay can be confirmed); then the verdicts.
4. If git_sha, analysis_version or a backend's runner_code_sha256 differs between the two reports, verdicts can differ even when the counts are identical, and a changed adapter can change that backend's output. Re-derive the verdicts on the new version rather than comparing labels.

Timing and memory methodology

• One warm-up, then the timed runs. Each backend runs the circuit once and discards the result, then runs it timing_repeats times (default 1) for the reported numbers. With one timed run every timing is a single sample (timing_samples = 1) and run-to-run variation is not measured. With more, compilation_time_sec and simulation_time_sec are medians (timing_statistic: "median"), and the per-run samples, mean and standard deviation are recorded beside them. execution_time_sec is the median compilation time plus the median simulation time; its IQR and standard deviation are those of the per-run totals. The statevector or counts and the memory fields come from the first timed run. In measured mode every timed run uses the same runner seed, so on a backend that honours the seed the repeats draw the same shots and only the timings differ. cpu_time_sec and process_peak_rss_mb cover every run, so they grow with the number of repeats.
• Timing windows. compilation_time_sec is wall clock from t0 to t1 (parse, transpile and, where the SDK allows, build the native circuit; see "Where circuit conversion falls"); simulation_time_sec is wall clock from t2 to t3 (execute, return the statevector or draw the shots). Building a counts histogram from drawn shots is outside the window.
• cuQuantum's timed window. cuQuantum's simulation_time_sec covers Aer's native execution of the assembled circuit: GPU memory allocation, the cuStateVec kernels and the copy of the result from device to host (Aer's conversion of the circuit is in compilation; see the next items). CUDA context initialisation happens in the warm-up run, outside the window; it shows up in cpu_time_sec, which covers the whole process.
• myQLM's timed window. myQLM returns a statevector as one Python object per non-zero amplitude (SAMPLE nbshots=0), and building that result is inside its simulation_time_sec and memory window, so its time and memory grow with the number of non-zero amplitudes rather than with 2^N alone: a GHZ state is fast, a dense state slow.
• Where circuit conversion falls. Compilation covers parsing, rebasing and optimisation passes, and building the executable object where the SDK lets the adapter build it separately (the PennyLane tape and device preprocessing, the myQLM and qibo circuit objects). For Qulacs, QuEST, qsim, Qiskit and cuQuantum the SDK's own call would convert the circuit inside the simulation (pytket's process_circuits, qsimcirq's simulate and run, Aer's run), so their adapters split that call at the native routine: the conversion (tk_to_qulacs, tk_to_quest, translate_cirq_to_qsim, Aer's assemble_circuits and run configuration) is in compilation_time_sec, and the timed run applies the converted circuit with the same routine, options and seeds as the SDK call. Old and new paths gave identical statevector bytes, identical counts under the same seed and identical executed texts in every case compared (scripts/adapter_paths/compare_paths.py): 1,080 to 2,200 cases per backend (a case is one circuit at one optimization level, with preprocessing on or off; a refusal counts only when both versions refused it the same way) from about 350 random circuits of up to 40 gates and 10 qubits, plus 24 statevector and 16 measured cases per backend from circuits of 500–3,000 gates on up to 16 qubits. On a 4-qubit, 2,200-gate circuit, simulation time fell from 28–37 ms to 1.3–2.3 ms in statevector mode (Qulacs, QuEST, qsim, Qiskit; cuQuantum 50 to 13 ms) and from 30–103 ms to 1.9–8.6 ms in measured mode (cuQuantum 47 to 18 ms), with totals within 3%. Two measured totals fell further: qsim's by up to 24%, because assembling its per-shot records from qsim's raw samples now follows the timed run, and Qulacs's by 17% on QFT-16, because the old call also copied and phase-corrected the whole statevector, which sampling does not use. Each adapter refuses to load with another version of the SDK whose call it reproduces. The disabled Braket and ProjectQ runners still convert inside the call.
• Different gate counts for the same circuit. Each backend executes the circuit after its own rebase, so the gate counts differ: at 18 qubits one circuit ran as 318, 750 or 1,158 gates depending on the backend. Simulation times compare those workloads; the Gate count column shows each one.
• Measured mode, statevector sampling. Backends that sample from a statevector (sampling_method: "sv_sample") read the state from the SDK and draw the shots inside simulation_time_sec, because the shots need the state; in statevector mode, converting the SDK's result object into the reported array is outside the window.
• Gate fusion. Qiskit Aer (qiskit and cuQuantum) and qsim fuse neighbouring gates inside their simulation at every optimization level: Aer above its qubit threshold (recorded per run as gate_fusion), qsim into gates on up to two qubits. The reported gate counts are those of the circuit before fusion. This is part of how those simulators run and is listed in their preprocessing_applied.
• quimb's timed window. quimb's simulation_time_sec includes the search for a contraction path (cotengra's auto-hq optimiser, repeated each run with its caches cleared) as well as the contraction. The search is randomised, so quimb's times vary between timed runs more than a state-vector backend's. It runs in the simulating process, with cotengra's worker pool turned off.
• CPU time. simulation_cpu_time_sec is process CPU time (all threads) over t2..t3 only. cpu_time_sec covers the whole isolated runner process: interpreter start, SDK import, warm-up and every timed run.
• What wall clock includes. Timings are wall clock, so they include time the process was not running: waits imposed by the CPU quota (recorded per window as simulation_throttled_sec and compilation_throttled_sec), preemption by other processes (counted in simulation_involuntary_switches_samples), and contention with the other backends of the same comparison, which run at the same time unless runner phases or the Sequential ordering stage them (each result records concurrent_runs, the number of runs dispatched in its phase, dispatch_phase, and overlapping_runs, the number of other runs of the same job that were in their runner process at some time this one was; other users' runs on a shared server are not counted). Concurrent runs share cores, hyperthreads and memory bandwidth: on a 20-qubit measured QFT testers saw 2.3–3.3× longer times than with the Sequential ordering, and the order of backends changed. For timings meant for a performance comparison, follow this protocol, which testers found repeatable on this host (medians within 0–3% between jobs for 13 of 14 backends, quimb excepted): run the comparison as at least three separate exclusive jobs, spaced apart, each with timing_repeats of at least 5; report the median of the job medians and the range between jobs; treat backends within about 3% of each other as tied; report quimb with its full range. The ≈ mark next to a rank (overlaps_next) means this backend's slowest timed run took at least as long as the next backend's fastest in the same job; it reflects the spread within one job only, so ranks without it can still swap between jobs. The "unstable" tag marks a run whose slowest timed run took more than 1.3× its fastest, from competing load or the backend's own variation. Use an exclusive run (the "Exclusive timing run" box, API field exclusive): the backends run one at a time and only while no other run is in flight on this server, and each result records exclusive and the wait (exclusive_wait_sec). Otherwise every local result records server_overlapping_runs, the runs of other jobs that were in flight (dispatched, including waiting in a runner's queue) at some time it was. On a host with hyperthreads, a run whose sibling thread is busy can be 1.5–2× slower with every other check passing, so a non-exclusive comparison is not reliable. Also use at least three timing_repeats (qibo's first timed run at 18 qubits was 14–17% slower in some jobs), stage the backends one at a time, and check that simulation_throttled_sec is zero and cpu_max_mhz names the same kind of core for every backend.
• Threads. Every runner is started with five thread variables set to 1: OMP_NUM_THREADS, MKL_NUM_THREADS, OPENBLAS_NUM_THREADS, NUMEXPR_NUM_THREADS and NUMBA_NUM_THREADS; the values each run saw are in thread_env. RUNNER_MAX_CONCURRENT (default 1) sets how many runs one runner executes at once; further runs queue.
• Memory the warm-up left behind. The warm-up run allocates and frees the same memory the timed run needs, and the allocator usually keeps it. The timed run can then reuse it without raising the high-water mark, so memory_usage_mb can read near 0 for a statevector of a few MiB (seen for PennyLane at 16 qubits and qibo at 18). Read it as the memory the simulation window added, not as the memory the simulation used. PennyLane-Lightning allocates its statevector when the device is created, during compilation, so its memory window holds only the copy read out at the end. A fixed allocator threshold that would expose every allocation slowed some backends by more than 3×, so it is not used.
• Memory. Memory is read from the kernel high-water mark (VmHWM). memory_usage_mb (Sim-window RSS delta) is the peak RSS minus the baseline RSS over the simulation window (pre-execute setup plus t2..t3; the high-water mark is reset when the window opens). process_peak_mb (Sim-window peak RSS) is the absolute peak over the same window. process_peak_rss_mb is the peak RSS of the simulating process over its whole life; Resources are ranked by it. memory_method and process_peak_rss_method record the method; where VmHWM is unavailable they read sampled_10ms (RSS polled every 10 ms, which can miss short peaks) and getrusage (ru_maxrss), and the UI shows a badge.
• Host memory only. Every memory field is host RSS. GPU device memory is not measured, so a GPU run's memory fields do not include its statevector on the GPU. GPU runs are not ranked by memory; they are listed under unranked with reason gpu_memory_not_measured, and the Performance tab says so.
• RSS and the container limit. RSS counts the shared libraries a process has mapped, such as the CUDA runtime, so process_peak_rss_mb can exceed the container's memory_limit_mib (cuQuantum and quimb read about 545 and 610 MiB under what was then a 512 MiB limit) without the process being out of memory.
• Units. Every memory field ending in _mb is in MiB (2²⁰ bytes).
• What dominates memory. For small circuits, process_peak_rss_mb is dominated by the Python interpreter and the SDK's import footprint, and is far larger than the statevector; compare it with theoretical_statevector_mb, the size of a dense statevector at the backend's precision: 2^N × 16 bytes for complex128, 2^N × 8 bytes for complex64 (theoretical_statevector_basis states which). quimb, a tensor-network backend, holds no dense statevector; its value is computed the same way from the precision it reports, for comparison. Differences between backends at small n reflect their libraries, not their simulation algorithms.
• Benchmark circuits and structure. The qft preset applies the QFT to |0…0⟩, so every controlled phase has its control in |0⟩ and the output is a uniform product state. Backends that exploit structure (tensor networks such as quimb, Qrisp's factorised state and gate grouping, sparse or decision-diagram methods) run it much faster than a dense statevector would suggest. For scaling studies, include circuits whose state is entangled and dense, such as random.
• Gate count and depth come from the circuit each backend executed: by default the top-level gate statements of its transpiled_qasm, with depth by greedy layering. The PennyLane runners count the operations of the tape the device executes instead, because their transpiled_qasm text decomposes operations such as QubitUnitary into gates that were not executed as such (a QubitUnitary on three or more wires is written as an opaque gate with its matrix in comments). gate_metrics_basis states which.
• Rankings include only successful runs. Failed runs, and GPU runs in the memory ranking, are listed under unranked with a reason, and the UI shows them after the ranked rows. The Performance tab's tables have a Device column, and the batch summary tags the device of the backend in its "Fastest (total time)" and "Lowest process peak RSS" columns.

Resources, queueing and timeouts

• CPU and memory quota. Each runner is limited to 1 CPU (a cgroup CPU quota), and its math libraries run one thread. Memory limits: 1 GiB per runner container in local mode (the docker-compose.yml limits) and per runner Job in Kubernetes mode; each result records the limit that applied as memory_limit_mib. A process that uses more than one core (an SDK thread pool the thread variables do not control) is throttled by the quota: the kernel stops it until the next 100 ms period, and that wait falls inside the timed window. Every result records the throttled time as simulation_throttled_sec and compilation_throttled_sec (read from the cgroup's cpu.stat just outside each window; null when the file is unreadable), and the Performance table tags runs held for at least 1% of the window "throttled". The cuQuantum runner's container has the same quota: its kernels run on the GPU, but its host-side work (circuit conversion, kernel launches, synchronisation, the device-to-host copy and sampling) is throttled like any other runner's.
• CPU context. Each result carries cpu_environment: the host CPU model, the quota in cores (cpu_quota_cores, null for none), the process's CPU affinity, and cpu_max_mhz for the logical CPUs the process was on at the start and end of each simulation window (simulation_cpus). On a host with cores of different speeds (for example Intel P- and E-cores) the same run can take different times depending on the core the operating system chose; cpu_max_mhz shows which kind it was. A server operator can restrict every runner process to chosen CPUs with the QROSETTA_CPU_AFFINITY environment variable (for example 0-15), recorded as cpu_environment.cpu_affinity_setting; memory_limit_mib records the runner container's memory limit. simulation_involuntary_switches_samples counts how often the process was preempted during each simulation window. simulation_cgroup_cpu_time_sec is the CPU time of every process in the runner's container over the simulation window; when it exceeds simulation_cpu_time_sec, part of the work ran outside the simulating process (for example in a worker pool), so the CPU time and memory fields leave that work out. The Performance table tags such runs "outside process".
• One run at a time per runner. A runner executes one run at a time. When several jobs, yours or other users', target the same runner, later runs wait for it. The wait is recorded as queue_wait_sec and is not part of any timing. Waits of 0.5 s or more are shown in the per-backend details and as a "queued" tag in the Performance tables; shorter values are dispatch overhead. In local mode a run may wait at most the server-wide cap RUNNER_QUEUE_MAX_WAIT_SEC (default 1800 s, recorded in the report as queue_max_wait_sec); a run still queued at the cap is not started. It ends with status timeout, timeout_kind: "queue" and never_started: true, and the Summary tab says it never started.
• Timeout scope. The timeout set in the Execution chip (the server's allowed range is shown under the field) applies to each backend separately and covers starting the isolated runner process, importing the SDK, the warm-up run and the timed run. In local mode the budget starts when the run leaves the runner's queue, so time spent waiting for a busy runner does not count against it; the queue wait has its own cap, independent of the timeout (see the previous item). In Kubernetes mode it runs from Job creation and also covers pod scheduling, image pull and the result upload. The report records the scope that applied. A backend that exceeds it ends with status timeout and timeout_kind: "run"; its process and every process it started (such as a compiler binary) are killed. A queue timeout (previous item) has timeout_kind: "queue". The value in the field is sent with every run from the page; API calls that omit timeout_seconds use the schema default shown in the Swagger UI (60 s for every run type). The field on the page starts at the same default.
• Timing samples. With the default of one timed run, a difference between two backends smaller than the run-to-run variation of either cannot be read from one report. Set Timing repeats above 1 to get a median and a standard deviation per backend, or use multi-run mode to compare separate runs.

Compile-only services

A compile-only service turns the run's circuit into another circuit and records the text. Its output is never simulated, has no timings, and is not part of the agreement comparison: compiled circuits are kept in the report's compiled_circuits, apart from raw_results. Tick levels in the Compile-only services chip's popover, or send compilers (compiler id → list of distinct levels, at most 15 compiles per circuit) with /compare, /compare_measured or /run_batch_suite; GET /compilers lists them with the versions each running compiler image reports (versions: pip SDK versions, native components built from source, build-file sha256 and image commit; null in Kubernetes mode, where no compiler runs between jobs). The config panel shows them under each compiler.

Every compiler receives the same circuit, recorded as compile_input: the submitted circuit as the API prepares it for statevector runners (terminal measurements removed, then the run's API preprocessing). In statevector mode this is the dispatched statevector circuit. A measured-mode circuit with mid-circuit measurement, reset or classical control is not compiled; each requested compile is then recorded with dispatch_status skipped and the reason. Compilers run after the run's simulators have finished, through the same queue and timeout as runners, so a compile never overlaps a timed run of its own job. They always run as a shared, not an exclusive, dispatch (they are not timed): an exclusive run of another client waits for them, and other jobs count a running compile in server_overlapping_runs. The report is complete only when every compile has finished or timed out, so a slow compile (BQSKit level 3 beyond a few qubits) delays it. Each compiler service runs two compiles at a time (each single-threaded, with its own CPU; BQSKit one, since its runtime binds fixed ports); a third waits in that service's queue, so a busy service can delay another client's compile; the live status grid lists each compile while it runs. A compile that fails or times out is recorded with its status and reason and does not affect the simulator results. A compile the adapter refuses (an input it cannot compile correctly, such as a cu3 Cirq would import wrongly) or whose output fails the adapter's check has refusal (the reason) and refusal_kind (input or output_check); the Summary lists every compile with its gate counts or reason, and a share link shows refusal reasons but no other error text.

The report records the request as compilers_requested. Each record's key is <compiler>~opt<level>~run1. It holds the output text (transpiled_qasm), the passes that produced it (passes_applied), gate count and depth counted from that text, the pip SDK versions (sdk_versions), the programs the image built from source (native_components: staq, Quartz and Synthetiq, with the source archive's URL and sha256, compiler flags or patches, and Quartz's ECC set hash), the sha256 of the image's build files (build_files) and the compiler's code hash. The Circuits tab shows the compiler input and each output; the RO-Crate holds them as compile_input.qasm and compiled_circuits/<key>.qasm, in a batch as compile_inputs/<task>.qasm and compiled_circuits/<task>/<key>.qasm. Each compile is a CreateAction from the compile input to its output in ro-crate-metadata.json, provenance.json lists every compile record under compile_only_services, and each native component is a SoftwareApplication of its own. For a compiler that produced output, its build files are included under compilers/<name>/ when the server's copy has the sha256 the image reported: the Dockerfile, requirements.lock, and for Quartz shift10-normalise.patch, the ECC generator gen_nam_ecc.cpp, the driver quartz_greedy.cpp and the ECC set Nam_complete_ECC_set.json. A file whose hash differs (the image was built from another commit) is left out. Records from images that report no build files get the lock file when its pins match the recorded SDK versions. Every crate's provenance.json records report_source: server_job for the server's copy of a job, upload or not_recorded for a saved run by how it was saved, and client_supplied for POST /export, whose contents (git commit, versions, circuit texts) are as the requester sent them. A field in the report cannot change it.

Outputs are deterministic on a given image: the same circuit, level and image give the same text in every job. BQSKit's numerical resynthesis is not portable beyond the image: on another Python or BLAS build its angles can differ in the last digits, while gate counts and two-qubit counts match. Each record keeps the image's runtime_environment, image_git_sha and compiler_code_sha256; cite the record's text rather than a regenerated text.

gate_count is the number of top-level gate statements in the output text and circuit_depth its greedy layering, one statement per layer slot whatever the gate; barriers are neither counted nor treated as aligning the wires, so the depth can be lower than Qiskit's depth() of the same text. The compilers write different gate sets: staq keeps ccx, cu1 and swap as single statements, Quartz writes h, x, rz and cx, Cirq cz and single-qubit gates. A smaller gate_count therefore does not mean a smaller circuit across compilers. Each record also has multi_qubit_gate_count (statements on two or more qubits) and gate_counts (statements per gate name); to compare compilers, translate the outputs to one gate set and count there. Compile-only services also run in experiments (compilers on POST /experiments). GUOQ (wisq) is not offered: its search stops on wall-clock time only (its seed covers resynthesis partitions), so its output would depend on machine load rather than on the circuit and settings.

Each compiler image has a self-test (python main.py --self-test) that compiles five probe circuits at every level (six for QMAP, Q-Synth and Quartz, which also compile a c3x followed by a qubit first used after it) and checks the output unitary against the input with the image's own SDK (BQSKit's unitary distance, PyZX's tensor comparison, Qiskit's operator equality with the circuit rebuilt from QMAP's // i and // o lines), and that the output declares as many qubits as the input. The probes include the circuits of the known issues below and a circuit with a creg.

With preprocessing on, compilers receive the pytket-rewritten circuit (see Circuit preprocessing); to see how a compiler treats your own text, run with preprocessing off. Several compilers then reject some valid OpenQASM 2.0, each recorded as that compile's error: BQSKit rejects register broadcasts (h q;), Cirq rejects barrier, the 4-parameter cu and rc3x, and Cirq, Q-Synth and Quartz reject u0, which Qiskit's reader does not take. Qiskit's translation for QMAP, Q-Synth and Quartz runs with qubits_initially_zero=False, so a multi-controlled gate is never decomposed with an unused qubit as a clean ancilla.

Reading the results

A run of the playlist shows a suite summary and the Exports tab. The suite summary gives each circuit an agreement badge plus one badge per failed-run status, and the fastest and lightest successful backends, each tagged with its device. A measured task is badged by measured_assessment: Differs, naming the backends, when any pair is distinguishable or a single-bit marginal differs, All pairs consistent when every pair is consistent within shot noise, otherwise Inconclusive. A statevector task is badged by label group: Differs when any pair disagrees (Differs, naming the backends), All pairs agree, or All pairs agree (bit order as declared) when some pairs agree after the declared bit reversal. Not compared means fewer than two backends returned a result; Compile only marks a task of a compile-only run. An experiment shows its pair counts, by assessment in measured mode and by label group in statevector mode, and the bundle download.

Reports and exports

The report sent to the browser replaces statevectors longer than 2¹⁴ amplitudes (circuits over 14 qubits) with a placeholder and sets statevector_omitted_for_display. The server keeps a finished job, with its untruncated report, for 30 to 40 minutes: jobs older than 30 minutes are removed by a sweep that runs every 10 minutes.

Export RO-Crate uses /jobs/{job_id}/export for a job run in this session (or the saved run, for a run opened from history) unless Browser copy is chosen in the Export panel, and offers the browser copy when the server copy has expired.

Accounts, history & autosave

An account is optional. Signing in (or registering with an email and password, via the Login / Register button) enables automatic run history for single runs and batches started from this page: each is saved to your account when it completes, under its run name. Experiments are never saved; their result is the bundle you download. Runs started through the API are not saved automatically; save one with POST /history/runs/from_job/{job_id} (see Reports and exports) while the server still holds it. Open Your Run History to browse past runs, rename them, download their stored reports (statevectors are removed before storage; their statevector_sha256 and statevector_length are kept), or delete them (singly or in bulk). Without an account, results exist only in your browser for that session.

The Circuits tab's Save and share box saves the run on screen to your history (also a run you started before signing in, while the server still holds it) and creates read-only links: Create link gives a /share.html#<token> address that expires after 1, 7, 30 or 90 days. An optional label names the link in your own list and is never shown to its recipients. A link created from a batch task's view opens on that task. Anyone with the address sees the run's name and save time and its circuit texts: the submitted circuit, the dispatched circuits and each backend's executed circuit with its transforms, SDK versions, gate count and depth, plus the compiler input and compiled circuits when the run had any. A report that its owner uploaded (POST /history/runs) instead of saving it from this server's job is marked as uploaded on the page. The page shows no results, timings, error messages or account details. The token is shown once; the server stores only its SHA-256, and the token sits after #, so browsers do not send it in requests or referrers. Revoke a link in the same tab. Deleting the run or the account deletes its links, and a deactivated account's links stop working.

A session lasts 30 minutes. If it expires before a run finishes, the page says the run was not saved and offers Sign in and save, which saves the run after you sign in again, as long as the server still holds the job. Delete Account, beside Logout in the header, removes the account with its saved runs and circuits; a session token issued before the deletion stops working.

Separately, and independent of an account, the Execution chip offers autosave to a local folder: each completed run is written as a JSON file directly to a folder on your computer, with nothing uploaded anywhere. This needs the File System Access API (Chromium-based browsers such as Chrome and Edge) and degrades gracefully elsewhere.

Limitations

• Agreement between backends is not correctness. Backends agreeing does not show they are all right, and a backend differing from the others does not show it is wrong. QRosetta has no independent reference to check against.
• Measured-mode thresholds are descriptive. The JS/Hellinger cut-offs are configurable and depend on the shot count; measured_assessment, from the statistical test, decides whether a pair disagrees.
• Permutation search is bounded. Exhaustive qubit-permutation matching is capped at 7 qubits. Larger cases report too_many_qubits instead of searching.
• Default timings are not a benchmark. By default each backend has one timed run and all backends run at the same time, competing for the host. Use the protocol in "Timing and memory methodology" (exclusive runs, several repeats and jobs) for comparisons. Results hold for one host, a 1-CPU quota and one thread.
• Pairwise flags can concentrate on one backend. Every pair that shares a backend reuses its sample, so one unusual sample can make several of that backend's pairs distinguishable at once. When all the flags of a run involve one backend at modest p-values, rerun with another seed or more shots before concluding.
• A residual disagreement is a starting point for investigation. None of the registered transforms explained the difference. The cause could be an unregistered convention, an environment limit, or a defect in a backend or its adapter.

Known backend issues

The discrepancies below come from the SDKs or their QASM importers, not from QRosetta. For simulators, the report shows them as disagreements (statevector) or distinguishable pairs (measured). For compile-only services, the rows say what the compiler's adapter does instead: refuse the circuit, avoid the defective call, or apply a recorded fix. QRosetta does not decide which side of a disagreement is wrong. Testers checked each case below against an independent exact simulator written from the qelib1.inc gate definitions; the table names the backend that differed from it. Check your own circuits the same way before attributing a disagreement. Every circuit starts with OPENQASM 2.0; include "qelib1.inc";.

Adding a simulator

Every runner implements the same contract: process_run (statevector) and process_run_measured (counts), built on the shared measurement instrument that does the timing, memory measurement, qubit-limit check and result assembly, so all backends are measured the same way. The step-by-step guide, including how to determine a backend's qubit ordering and how to register it, is CONTRIBUTING.md in the repository.

Glossary

Agreement classes

See the label tables in Agreement classification.

Not-comparable reasons (metrics.reason)

Statistical verdicts (measured mode)

Unranked reasons (Performance / Resources)

Run statuses

FAQ

Why would two correct simulators disagree?

Usually because of a convention: a different declared qubit ordering or an unobservable global phase. The agreement classification accounts for these so that a convention difference is reported as agreement. A difference that no declared convention explains is reported as a disagreement; Known backend issues lists the ones testers have traced.

Is a "distance exceeds threshold" pair a bug?

Not by itself. It means the distances stayed above the descriptive thresholds after every registered transform. Check the pair's measured_assessment: if it is consistent_within_shot_noise, the difference is what sampling alone produces at this shot count; if inconclusive_low_shots, rerun with more shots when resolvable_by_more_shots is true; only distinguishable indicates a real difference, whose cause could be an unregistered convention, a resource limit, or a defect in a backend or its adapter.

Does QRosetta modify a backend's output?

No. Outputs are stored as each backend returned them. Bit reversal, phase alignment and permutation matching are analysis steps reported with their numerical effect. The input circuit can be changed before dispatch; every such change is listed in preprocessing_applied (see Circuit preprocessing).