V8's Performance Boost: Mutable Heap Numbers Unveiled

At V8, the constant pursuit of JavaScript speed led to a deep dive into the JetStream2 benchmark suite. One crucial optimization emerged, slashing execution time by 2.5x in the async-fs benchmark. This article breaks down the discovery and its impact, focusing on how mutable heap numbers transformed a hidden performance cliff into a tangible gain. Let’s explore the key questions.

What is the async-fs benchmark and why does it matter?

The async-fs benchmark simulates a JavaScript file system with asynchronous operations. It’s part of the JetStream2 suite, which measures real-world JavaScript performance. Surprisingly, the bottleneck wasn’t file I/O but a custom Math.random implementation used for consistent, deterministic results across runs. This function updates a seed variable with arithmetic and bitwise operations, called repeatedly during the benchmark. The seemingly minor code hidden inside Math.random caused major slowdowns, revealing how even standard patterns can hide performance traps in JavaScript engines.

How is the seed variable stored in V8?

In the given implementation, seed is stored in a ScriptContext, an internal array holding tagged values accessible within a script. On 64-bit V8, each entry occupies 32 bits. The least significant bit acts as a tag: 0 for a 31-bit Small Integer (SMI), 1 for a compressed pointer to a heap object. For numbers too large or fractional, V8 uses an immutable HeapNumber—a 64-bit double allocated on the heap. The ScriptContext then holds a pointer to that HeapNumber. This allows efficient handling of both small integers and larger numeric values, but as we see, it can become a bottleneck when the value changes frequently.

What is the performance bottleneck with this storage scheme?

Profiling revealed two critical issues. First, each time Math.random updates seed, a new HeapNumber must be allocated on the heap because the existing one is immutable. This allocation happens on every call, generating significant garbage. Second, the ScriptContext’s pointer must be updated to the new HeapNumber. Together, these operations waste CPU cycles and memory bandwidth. For a function called thousands of times per second, the overhead becomes substantial, dragging down overall benchmark scores—and potentially any real-world code that follows a similar pattern (e.g., storing mutable state in a closure with non-SMI values).

What are SMIs and HeapNumbers, and why does the distinction matter?

SMIs (Small Integers) are 31-bit signed integers stored directly inside the tagged slot—no heap allocation needed. They cover values from -2^30 to 2^30-1. HeapNumbers, on the other hand, are 64-bit doubles placed on the heap, requiring memory allocation and pointer indirection. The seed variable in the example undergoes arithmetic and bitwise operations that quickly push it beyond the SMI range (e.g., after shifts and additions, it can exceed 31 bits). Consequently, V8 treats it as a HeapNumber, triggering repeated allocations. This distinction is crucial: while SMIs offer cheap updates, HeapNumbers incur allocation costs every time the value changes, which directly impacts performance in tight loops.

Does this pattern appear in real-world code beyond benchmarks?

Yes. The Math.random implementation in the async-fs benchmark is contrived for reproducible results, but similar patterns exist in production. For instance, any code that maintains a numeric accumulator updated in a hot path—counters, statistical sampling, PRNGs, or progressive filters—can fall into the same trap. Even basic state machines with numeric flags stored in closures may cause HeapNumber allocations if the values grow beyond SMI range. The V8 team noted that while this optimization was benchmark-inspired, the underlying pattern is real. Developers can avoid this by keeping state within SMI range where possible, or by relying on engine improvements like mutable heap numbers that V8 later introduced.

How did V8 address the HeapNumber allocation issue?

While the original post described the bottleneck, V8’s solution was to introduce mutable heap numbers. Instead of allocating a new HeapNumber on every update, the engine modifies the existing HeapNumber object in place. This requires careful handling of garbage collection and concurrency, but eliminates the allocation overhead. The ScriptContext pointer remains unchanged, and the double value is directly updated. For the async-fs benchmark, this change yielded a 2.5x speedup. The optimization also improved other benchmarks and real-world applications that repeatedly update non-SMI numeric values stored in heap objects, such as those in arrays or object properties.

What’s the key takeaway for JavaScript developers?

Understanding V8’s internal representation can help write more efficient code. While engine optimizations like mutable heap numbers reduce the impact of naive patterns, developers can still benefit from keeping mutable numeric state within SMI range (values between -2^30 and 2^30-1). If that’s not possible, consider using Float64Array or other typed arrays for predictable performance. More importantly, profiler-guided optimizations remain essential—tools like Chrome DevTools can reveal unexpected allocation hotspots. The V8 team’s work shows that even small changes in the engine can dramatically improve real-world performance, and developers should stay informed about such advancements to leverage them in their applications.