Benchmark explorer

Each benchmark uses its full measured cohort's fixed fewest-lines and fastest-median baselines. Data filters change eligible languages, benchmarks and programs without moving those baselines. Samples, Medians and Density are independent visual layers with Off, Dim and Highlight modes. Dim has 15% opacity; Highlight has full contrast. Modes remain selected during source navigation and Back.

Samples shows actual recorded runs at the selected source measure divided by its source baseline and elapsed seconds divided by the time baseline. Explicit timing aliases share one set of recorded runs; source aliases with independent timings and coincident recorded values remain separate observations. Medians controls profile points, benchmark medians, links, labels and covariance. Circles select fastest programs; squares select smallest source under the selected measure. Global median & SD independently shows coordinate medians and covariance at full contrast. It remains visible when Medians is Off or Dim, so the chart can show only the global summary.

Density counts the same eligible recorded runs, whether their dots or profile medians are visible. Each recorded run has weight one; shared timing aliases count once and independently recorded equal values remain separate. A Gaussian field covers a 48 × 36 grid across the chosen log₂ plot domain, with bandwidth 1.25 cells on each axis. Colors show relative square-root-scaled intensity, not a probability-normalized score. The legend reports recorded runs, measured programs, variants and smoothing. Ordinary views use the admitted program pool; highlighted language overviews use their qualified full source catalog.

Axes stays available independently of all layers. Medians fits profile summaries; All samples fits every recorded run and complete covariance envelopes. Language overviews include their full source catalog and start with All samples. Density fills whichever range you choose. Only your framing choice, source metric, chart view, source focus or selected benchmark set changes the range; layer, language, year, profile and program visibility leave it fixed.

The Source metric selector offers retained normalized application LOC, qualified formatted LOC, qualified logical units with partial coverage, application UTF-8 bytes, deterministic gzip bytes and pinned cl100k_base BPE tokens. The retained normalized counts have known statement and literal-wrapping gaps. Logical units uses exact-source native parse evidence with versioned owner rules: Python/Julia v2 and Clojure v1. Independent statements/bindings count separately; array row wrapping and closing syntax contribute no units. Policy-specific conditional, definition and loop choices remain explicit; this is not a whole-language certificate or a uniform fairness claim. Formatted LOC instead counts nonblank, noncomment lines after the language project's native formatter, across application source and workload-specific helpers. Missing or incomplete proof leaves the corresponding total and coordinate blank. Text-size measures include original comments and whitespace within the exact application partition. Logical and text-size measures each use a fixed per-workload minimum over all eligible, qualified timed candidates, and can select different fastest and smallest programs from that inventory. gzip depends on formatting/repetition and BPE on its vocabulary and Unicode spelling; neither supplies a language-neutral logical-source score. No missing measure borrows another measure's value. Timings, recorded runs and the fixed best-time baseline retain their original values. Unsupported programs remain available in language overviews and source details; available submitted and excluded support counts remain inspectable and exportable.

Runtime / toolchain filters group the measured implementations as Native toolchains, JVM, .NET, BEAM, Node.js / V8, or Other language runtimes. .NET includes JIT and NativeAOT; Node.js / V8 includes JavaScript and Elm. Every published variant has an explicit classification, with its exact runtime retained in the language metadata. Execution mode is a separate filter. Historical version, flag, or executable evidence can be incomplete without making the recorded runtime family unknown. Visibility and LOC metric changes preserve these classifications.

Language checkboxes group runtime variants: Mech has AOT, JIT and interpreted execution; Lua has interpreted Lua and LuaJIT; C# and F# each have JIT and AOT; Dart has AOT and JIT. Child checkboxes keep each variant independently selectable, with separate measured endpoints and means. A partially checked parent means some variants are enabled. Lua has nine admitted workloads and no recorded pi-digits endpoint; its partial coverage is labelled, with no invented timing.

The comparison table shows each language’s smallest-source multiple under the selected measure and fastest-time multiple relative to 1×. On the full chart these are equal-weight geometric means over the compared benchmarks. A 1.46× source value means 46% more measured source than the smallest-source baseline. These two best values can come from different programs. For the line from the square to the circle, Δx = log₂(fastest source size / smallest source size) and Δy = log₂(fastest time / smallest-source program time). Slope = Δy/Δx; length = √(Δx² + Δy²), in doublings rather than screen pixels. Coincident endpoints have zero length and undefined slope.

The release-year window includes both selected endpoints. Earlier languages are excluded from the visible comparison. Playback steps only through matching years inside that window. Mech uses 2026 for the version under test. Language years and their sources are listed in the controls. This changes visibility only: benchmark samples, geometric-mean inputs, normalization baselines and axis limits stay tied to the selected benchmark cohort.

Hovering a profile point or line shows its covariance contours; oval controls keep them visible within the Medians layer. Contours use sample covariance in log₂ source/time coordinates, centered at geometric means. Inner contours are 1σ; outer contours have twice the radius. Whole-set coordinate-median contours preserve the covariance shape and area. They describe variation across benchmarks or variants, not run uncertainty; program details retain separate run-to-run SD. Rankings uses the same exact coordinate medians and log-space SD; display summaries use one decimal while underlying data and program timing precision remain unchanged.

Apple M3 · AC power · Low Power Mode off · October 2026. Three samples in broad comparisons; five, seven, or ten in subsequent measurements and confirmations. All ranked times run from process launch through exit, including startup, input and complete output. Mech JIT includes parsing and compilation; Mech interpreted includes loading and preparation. Offline AOT builds and post-run validation are outside the timed interval. Internal phase timings are diagnostic measurements and do not enter the rankings. Retained normalized LOC excludes blank and comment-only lines and uses cohort-specific statement/assignment adjustments; those rules have known gaps and are not one uniform logical-LOC grammar. Workload-specific application source and helpers count. Reusable hosts, generic FFI bindings and general libraries are excluded, including reviewed sections embedded in application files. Source details and exports retain previous totals, the support-code breakdown and separate exact-text byte/gzip/BPE measurements.

Mech includes AOT, JIT, and interpreted execution of the same audited source candidates. Solid, dashed, and dotted lines distinguish these paths; the Mech view also uses three shades of yellow. Each path selects its fastest and fewest-LOC endpoints from its own measurements. A program’s final statement is its output. Known work mismatches are excluded before selecting endpoints; correct output alone is insufficient. Library allocation strategies are recorded. Fastest means the best eligible measured program in this cohort. Each language’s geometric means use its measured selected benchmarks. Partial suites are labelled with their coverage, such as 9/10, on the line and in the table; missing or failed tests contribute no invented result. BQN and K cover all ten full workloads; their faithful spectral-norm and k-nucleotide sources replace the archived shortcuts.

Downloadable cohort reports below preserve their historical source counts and measurements. The chart, Data view and Data exports use the corrected application-LOC policy.

MATLAB R2026b covers all ten full workloads with twenty local compact/optimized source candidates and three complete, validated fresh-process samples per source. Endpoints use the measured winners; an optimized candidate is not assumed to be faster. Each algorithm remains in the displayed MATLAB source. Base MATLAB standard JIT and automatic computation threads are used, with no Parallel Computing Toolbox or Coder. General Java BigInteger supplies pi arithmetic. Startup, cached online-license checks, first-use JIT, complete input, computation, output and teardown are timed. The source selector shows the stream-only input launcher separately from counted application code. Untimed identity probes verify independent two-slot allocations for tree interiors and leaves through both even- and odd-depth constructor branches. Current unchanged Mech AOT controls were measured alongside MATLAB; earlier chart samples are retained.

Haskell and OCaml cover all ten full workloads with three local samples per admitted source. The Functional group includes Haskell, OCaml and Futhark. Published sources are retained or explicitly labelled when corrected; all corrections count toward LOC. Packed independent statements and bindings count separately, and semantic compiler pragmas count as code. Tree checks preserve per-node allocation and lifetime; nucleotide maps include every window and grow from small initial tables. Full source and sample timings are available by clicking their points.

The current Mech JIT and interpreted cohort reruns all source candidates after general execution improvements. Native admission shares one parsed document, releases it after admission, and reuses the admitted control graph. The standard counting container keeps its batch dispatch inlined while retaining every input update. Interpreted matrix products batch the source coefficient instructions across independent elements, retain checked-arithmetic errors and scalar sum order, and recompute every coefficient for each product. Fifty-four semantic checks and fifteen byte-identical AOT objects accompany the full-workload output checks. Earlier timings remain archived.

Swift covers all ten workloads. All 37 published sources were considered; 35 admitted candidates have three complete fresh-process samples. Ten local corrections are labelled in the code panel: API compatibility, common floating-point FASTA work, or required tree lifetimes. Originals are retained in the audit. Builds use Apple Swift 5.10 with the published optimization flags; published Linux timings are not reused. Current unchanged Mech AOT programs were timed alongside Swift as controls. Existing Mech cohort measurements are retained.

Rust’s fastest FASTA source is a local adaptation of published Rust #7. It retains the threaded pipeline and uses unscaled floating-point probabilities and direct division. Five full-size runs verify its complete output; the unchanged Mech FASTA binary was measured in the same rounds. The code panel identifies the adaptation and shows its complete source. Language groups can overlap; execution groups describe the timed implementations, including OpenJDK JVM/JIT for Java. Runtime, execution, typing, paradigms and memory models are independently audited.

R, GNU APL and Uiua use full canonical workloads with three fresh-process samples per admitted source and fresh unchanged Mech AOT controls. Partial coverage is shown explicitly; failed or pending workloads do not contribute timings. Compact and optimized forms retain the algorithm in the language, with ordinary GMP and regex bindings where required. Source panels show seconds, ratios to the benchmark’s fixed best-time baseline, and normalized LOC.

The interpreter integration audit completed 90 full-workload runs across 15 unchanged Mech sources. The interpreted rankings use a separate fresh run of all 15 sources, three whole-process samples each, including startup and preparation. The linked before/after study compares internal execution phases only. Full prior measurements remain archived, and AOT/JIT timings are unchanged. LLVM codegen partitioning and generated control-interpreter code changed even though these workloads do not call the new f32 executor. Default fannkuch measured 5.93% slower in the main cohort and 5.67% slower in independent confirmation. Confirmation samples are retained separately, and the cause of the slowdown is not proven. Before/after timings (CSV) · Integration audit.

The latest n-body audit includes matched C++, Chapel, Rust and Zig controls. Two seven-round cohorts confirm a 1.4% improvement for the unchanged 25-LOC Mech source using square-reduction simplification and general graph scheduling (1.054 s to 1.039 s). Zig takes 1.007 s. The unchanged compact source retains its previous scheduling improvement and takes 1.090 s at 20 LOC. All execute every pair on all 50 million steps. Matched timings and LOC · Compiler and work audit.

The remaining Benchmark Game cohort includes C#, F#, Ada, Free Pascal, Fortran, Dart, and Smalltalk. All use the same ten prescribed workloads with three sequential fresh-process samples per admitted candidate and fresh unchanged Mech AOT controls. Published sources are preserved, with local ARM64 portability and required-work corrections shown in each code panel. Smalltalk runs in a headless Pharo JIT image; Fortran uses GNU Fortran rather than the published Intel compiler. Generic arithmetic and regex libraries are allowed; the benchmark control and application helpers remain in the named language. Timing tables (CSV) · Readable tables.

Mojo covers all ten full CPU workloads; Futhark covers nine. Twenty distinct compact/performance pairs were attempted, with nineteen pairs and fifty candidate sources admitted after review. At most eight workers and three fresh samples per source. Endpoints select measured winners across this pool. Matching community Mojo N-body and Spectral Norm sources were adapted; the remaining Mojo programs and all Futhark programs are local ports. Each algorithm executes in its language source. Generic IO, containers, arithmetic, regex and APR pool support is pinned, independently tested and inspectable in the source selector. Corrected Mojo trees allocate sixteen bytes for every node through the Apache APR library pool, as in the C reference; test instrumentation confirms every allocation. Earlier collapsed loops and array-allocation mismatches are archived and excluded under the official tree allocation rules. Stock C and multicore allocation probes confirm that non-inlined per-node arrays are copied into flat storage and released immediately, failing the required lifetime. A faithful tree program needs a general heap-reference or allocator compiler extension. Futhark appears on the full chart with a 9/10 label, using its nine admitted workloads. Other languages retain their ten-test means. Current Mech controls use the original arguments and runtime settings.

Zig, Odin, Roc and Elm each cover ten full workloads. Fastest and fewest-LOC endpoints select the validated measured winners. Workload-specific helpers count toward application LOC; reusable FFI bindings, library implementations and generic process I/O support are excluded. Roc uses the established alpha4 LLVM compiler from September 2025; Elm uses optimized JavaScript under Node. Their source panels show raw seconds and ratios to the benchmark’s fixed best-time baseline. Matched Mech controls remain in the linked timing tables, and the chart retains the earlier Mech cohort measurements. Download all four comparisons to Mech (CSV) · Read the full timing and LOC tables.

Chapel benchmark comparison