How to Evaluate WebAssembly Performance Using JetStream 3: A Step-by-Step Guide

Introduction

Benchmarking is essential for driving browser engine performance, but as the web evolves, benchmarks must too. JetStream 3, a major update developed collaboratively by WebKit, Google, and Mozilla, addresses critical flaws in its predecessor—particularly for WebAssembly (Wasm). This guide walks you through the key changes in JetStream 3 and explains how to interpret its results to better understand modern web performance. Whether you're a browser engineer or a performance enthusiast, these steps will help you leverage the new benchmark effectively.

What You Need

• A basic understanding of browser benchmarks and WebAssembly.
• Access to the JetStream 3 benchmark suite (official page).
• A modern browser that supports WebAssembly (e.g., latest Chrome, Firefox, Safari).
• Familiarity with JavaScriptCore or other engine debugging tools (optional).

Step-by-Step Guide

Step 1: Understand the Evolution of WebAssembly Benchmarking

JetStream 2 was released when WebAssembly was still emerging. It measured Wasm in two distinct phases: Startup (time to instantiate a module) and Runtime (throughput after startup). Initially, this made sense—large C/C++ applications often tolerated long startup costs for high runtime performance. However, browser engines rapidly improved startup times. In WebKit, optimizations for the instantiation path made certain small workloads complete in under 1 millisecond, effectively reaching zero. This created a problem: JetStream 2 used Date.now() which rounds down, so sub‑1ms times became 0 ms.

Step 2: Recognize the 'Infinity Problem' in JetStream 2

Because JetStream 2 scored each iteration as Score = 5000 / Time, a 0 ms time produced an infinite score. This made the sub‑test dominant and rendered the overall benchmark meaningless. The issue was patched in JetStream 2.2 by clamping the score to 5000, but the underlying flaw remained: zero startup times in microbenchmarks didn’t reflect real‑world scenarios where Wasm is on the critical path—for example, in libraries, image decoders, and UI frameworks.

Step 3: Discover How JetStream 3 Redesigns Wasm Workloads

JetStream 3 fundamentally shifts from separate Startup and Runtime phases to a single, integrated measurement. Instead of artificial micro‑tasks, it uses workloads that mirror actual web applications—larger, more complex modules that require both efficient startup and sustained runtime performance. This eliminates the possibility of zero‑time anomalies and provides a holistic view of Wasm performance. The new suite also includes more diverse workloads, such as video games and scientific simulations, to better represent modern uses.

Step 4: Learn the New Scoring Methodology

The scoring in JetStream 3 is designed to reward balanced improvements. Instead of infinite scores from miniscule startup gains, each sub‑test now uses a robust timing mechanism (e.g., high‑resolution performance timers) and a logarithmic‑like scale that prevents extreme outliers from dominating. The overall suite score is a geometric mean of all sub‑tests, ensuring that no single workload disproportionately affects the result. This makes the benchmark more resistant to over‑optimization for specific patterns.

Step 5: Apply JetStream 3 to Real-World Performance Analysis

Run JetStream 3 in your browser and compare results across different engines. Pay attention to the Wasm subtests—they are now better proxies for how Wasm behaves in large applications. Use the data to guide optimizations: for example, if your engine scores low on a particular video game workload, focus on improving both instantiation and execution for that pattern. Remember that JetStream 3 is a collaborative effort; its workloads reflect input from multiple vendors, so improvements are likely to be general rather than engine‑specific.

Step 6: Monitor Ongoing Evolution

Benchmarks are never final. Just as JetStream 2 became outdated, JetStream 3 will eventually need updates. Follow the discussions at the WebKit, Google, and Mozilla blogs to stay informed about future revisions. Consider contributing feedback on the benchmark suite to improve its coverage of emerging patterns like streaming Wasm compilation or threaded modules.

Tips for Effective Use

• Focus on realistic workloads: JetStream 3’s strength is its real‑world relevance. Avoid cherry‑picking only the subtests where your engine excels.
• Combine with other benchmarks: Use JetStream 3 alongside Speedometer, MotionMark, and other suites for a holistic view.
• Watch for over‑optimization: Because benchmarks drive engineering decisions, be cautious about optimizing specifically for JetStream 3 patterns—ensure gains translate to general web pages.
• Leverage the collaborative design: The fact that multiple vendors contributed means improvements made to satisfy JetStream 3 are likely to benefit users across all browsers.