Benchmark explorer

Each benchmark uses its full measured cohort's fixed fewest-lines and fastest-median baselines. Data filters change eligible languages, benchmarks and programs without moving those baselines. Samples, Medians and Density are independent visual layers with Off, Dim and Highlight modes. Dim has 15% opacity; Highlight has full contrast. Modes remain selected during source navigation and Back.

Samples shows actual recorded runs at source LOC divided by the source baseline and elapsed seconds divided by the time baseline. Coincident recorded values remain separate observations. Medians controls profile points, benchmark medians, links, labels and covariance. Circles select fastest programs; squares select fewest LOC. Global median & SD adds coordinate medians, covariance and release history within that layer, so Medians Off also removes these summaries. Changing global statistics never overrides a layer's selected opacity.

Density counts the same eligible recorded runs, whether their dots or profile medians are visible. Each distinct program/run has weight one; repeated equal values are retained. A Gaussian field covers a 48 × 36 grid across the chosen log₂ plot domain, with bandwidth 1.25 cells on each axis. Colors show relative square-root-scaled intensity, not a probability-normalized score. The legend reports actual runs, programs, variants and smoothing. Ordinary views use the admitted program pool; highlighted language overviews use their qualified full source catalog.

Axes stays available independently of all layers. Medians fits profile summaries; All samples fits every recorded run and complete covariance envelopes. Language overviews include their full source catalog and start with All samples. Density fills whichever range you choose. Only your framing choice, chart view, source focus or selected benchmark set changes the range; layer, language, year, profile and program visibility leave it fixed. Data loads revalidate data.json with the server so a normal reload can receive newly published cohorts.

Language checkboxes group runtime variants: Mech has AOT, JIT and interpreted execution; Lua has interpreted Lua and LuaJIT; C# and F# each have JIT and AOT; Dart has AOT and JIT. Child checkboxes keep each variant independently selectable, with separate measured endpoints and means. A partially checked parent means some variants are enabled. Lua has nine admitted workloads and no recorded pi-digits endpoint; its partial coverage is labelled, with no invented timing.

The comparison table shows each language’s fewest-LOC multiple and fastest-time multiple relative to 1×. On the full chart these are equal-weight geometric means over the compared benchmarks. A 1.46× LOC value means 46% more lines than the shortest-source baseline. These two best values can come from different programs. For the line from the square to the circle, Δx = log₂(fastest LOC / fewest LOC) and Δy = log₂(fastest time / fewest-LOC time). Slope = Δy/Δx; length = √(Δx² + Δy²), in doublings rather than screen pixels. Coincident endpoints have zero length and undefined slope.

The release-year window includes both selected endpoints. Earlier languages are excluded from the visible comparison and its median history. Playback steps only through matching years inside that window. Mech uses 2026 for the version under test. Language years and their sources are listed in the controls. This changes visibility only: benchmark samples, geometric-mean inputs, normalization baselines and axis limits stay tied to the selected benchmark cohort.

Hovering a profile point or line shows its covariance contours; oval controls keep them visible within the Medians layer. Contours use sample covariance in log₂ LOC/time, centered at geometric means. Inner contours are 1σ; outer contours have twice the radius. Whole-set coordinate-median contours preserve the covariance shape and area. They describe variation across benchmarks or variants, not run uncertainty; program details retain separate run-to-run SD. Rankings uses the same exact coordinate medians and log-space SD; display summaries use one decimal while underlying data and program timing precision remain unchanged.

Apple M3 · AC power · Low Power Mode off · October 8–11, 2026. Three samples in broad comparisons; five, seven, or ten in subsequent measurements and confirmations. Fresh-process times include startup and full output. Mech JIT also includes compilation. Mech interpreted measures loaded execution, including input and output, with loading and activation excluded; its code panel also records whole-process times. LOC excludes blank and comment-only lines. Packed independent statements and assignments are counted separately in every language. Complete application source, including helpers and FFI declarations, is counted; general libraries and compilers are excluded.

Mech includes AOT, JIT, and interpreted execution of the same audited source candidates. Solid, dashed, and dotted lines distinguish these paths; the Mech view also uses three shades of yellow. Each path selects its fastest and fewest-LOC endpoints from its own measurements. A program’s final statement is its output. Known work mismatches are excluded before selecting endpoints; correct output alone is insufficient. Library allocation strategies are recorded. Fastest means the best eligible measured program in this cohort. Each language’s geometric means use its measured selected benchmarks. Partial suites are labelled with their coverage, such as 9/10, on the line and in the table; missing or failed tests contribute no invented result. BQN and K cover all ten full workloads; their faithful spectral-norm and k-nucleotide sources replace the archived shortcuts.

MATLAB R2026b covers all ten full workloads with twenty local compact/optimized source candidates and three complete, validated fresh-process samples per source. Endpoints use the measured winners; an optimized candidate is not assumed to be faster. Each algorithm remains in the displayed MATLAB source. Base MATLAB standard JIT and automatic computation threads are used, with no Parallel Computing Toolbox or Coder. General Java BigInteger supplies pi arithmetic. Startup, cached online-license checks, first-use JIT, complete input, computation, output and teardown are timed. The source selector shows the stream-only input launcher separately from counted application code. Untimed identity probes verify independent two-slot allocations for tree interiors and leaves through both even- and odd-depth constructor branches. Current unchanged Mech AOT controls were measured alongside MATLAB; earlier chart samples are retained.

Haskell and OCaml cover all ten full workloads with three local samples per admitted source. The Functional group includes Haskell, OCaml and Futhark. Published sources are retained or explicitly labelled when corrected; all corrections count toward LOC. Packed independent statements and bindings count separately, and semantic compiler pragmas count as code. Tree checks preserve per-node allocation and lifetime; nucleotide maps include every window and grow from small initial tables. Full source and sample timings are available by clicking their points.

The current Mech JIT and interpreted cohort reruns all source candidates after general execution improvements. Native admission shares one parsed document, releases it after admission, and reuses the admitted control graph. The standard counting container keeps its batch dispatch inlined while retaining every input update. Interpreted matrix products batch the source coefficient instructions across independent elements, retain checked-arithmetic errors and scalar sum order, and recompute every coefficient for each product. Fifty-four semantic checks and fifteen byte-identical AOT objects accompany the full-workload output checks. Earlier timings remain archived.

Swift covers all ten workloads. All 37 published sources were considered; 35 admitted candidates have three complete fresh-process samples. Ten local corrections are labelled in the code panel: API compatibility, common floating-point FASTA work, or required tree lifetimes. Originals are retained in the audit. Builds use Apple Swift 5.10 with the published optimization flags; published Linux timings are not reused. Current unchanged Mech AOT programs were timed alongside Swift as controls. Existing Mech cohort measurements are retained.

Rust’s fastest FASTA source is a local adaptation of published Rust #7. It retains the threaded pipeline and uses unscaled floating-point probabilities and direct division. Five full-size runs verify its complete output; the unchanged Mech FASTA binary was measured in the same rounds. The code panel identifies the adaptation and shows its complete source. Language groups can overlap; execution groups describe the timed implementations, including GraalVM native AOT for Java.

R, GNU APL and Uiua use full canonical workloads with three fresh-process samples per admitted source and fresh unchanged Mech AOT controls. Partial coverage is shown explicitly; failed or pending workloads do not contribute timings. Compact and optimized forms retain the algorithm in the language, with ordinary GMP and regex bindings where required. Source panels show seconds, ratios to the benchmark’s fixed best-time baseline, and normalized LOC.

The interpreter integration audit completed 90 full-workload runs across 15 unchanged Mech sources. Interpreted chart points use all 45 canonical after samples; full prior measurements remain archived, and AOT/JIT timings are unchanged. LLVM codegen partitioning and generated control-interpreter code changed even though these workloads do not call the new f32 executor. Default fannkuch measured 5.93% slower in the main cohort and 5.67% slower in independent confirmation. Confirmation samples are retained separately, and the cause of the slowdown is not proven. Before/after seconds and × Mech AOT (CSV) · Integration audit.

The latest n-body audit includes matched C++, Chapel, Rust and Zig controls. Two seven-round cohorts confirm a 1.4% improvement for the unchanged 25-LOC Mech source using square-reduction simplification and general graph scheduling (1.054 s to 1.039 s). Zig takes 1.007 s (0.968× Mech). The unchanged compact source retains its previous scheduling improvement and takes 1.090 s at 20 LOC. All execute every pair on all 50 million steps. Matched seconds, × Mech and LOC table · Compiler and work audit.

The remaining Benchmark Game cohort includes C#, F#, Ada, Free Pascal, Fortran, Dart, and Smalltalk. All use the same ten prescribed workloads with three sequential fresh-process samples per admitted candidate and fresh unchanged Mech AOT controls. Published sources are preserved, with local ARM64 portability and required-work corrections shown in each code panel. Smalltalk runs in a headless Pharo JIT image; Fortran uses GNU Fortran rather than the published Intel compiler. Generic arithmetic and regex libraries are allowed; the benchmark control and application helpers remain in the named language. Seconds and × Mech tables (CSV) · Readable tables.

Mojo covers all ten full CPU workloads; Futhark covers nine. Twenty distinct compact/performance pairs were attempted, with nineteen pairs and fifty candidate sources admitted after review. At most eight workers and three fresh samples per source. Endpoints select measured winners across this pool. Matching community Mojo N-body and Spectral Norm sources were adapted; the remaining Mojo programs and all Futhark programs are local ports. Each algorithm executes in its language source. Generic IO, containers, arithmetic, regex and APR pool support is pinned, independently tested and inspectable in the source selector. Corrected Mojo trees allocate sixteen bytes for every node through the Apache APR library pool, as in the C reference; test instrumentation confirms every allocation. Earlier collapsed loops and array-allocation mismatches are archived and excluded under the official tree allocation rules. Stock C and multicore allocation probes confirm that non-inlined per-node arrays are copied into flat storage and released immediately, failing the required lifetime. A faithful tree program needs a general heap-reference or allocator compiler extension. Futhark appears on the full chart with a 9/10 label, using its nine admitted workloads. Other languages retain their ten-test means. Current Mech controls use the original arguments and runtime settings.

Zig, Odin, Roc and Elm each cover ten full workloads. Fastest and fewest-LOC endpoints select the validated measured winners. All application helpers and FFI declarations count toward LOC; ordinary library implementations and generic process I/O support are excluded. Roc uses the established alpha4 LLVM compiler from September 2025; Elm uses optimized JavaScript under Node. Their source panels show raw seconds and ratios to the benchmark’s fixed best-time baseline. Matched Mech controls remain in the linked timing tables, and the chart retains the earlier Mech cohort measurements. Download all four comparisons to Mech (CSV) · Read the full timing and LOC tables.

Chapel benchmark comparison