Once again I’ve had to purchase more memory after swapping brought my Thinkpad X250 to a grinding halt. As I was waiting for it to finally be delivered, I decided to take a closer look into what is actually using all this memory.

 

I already have 8GB in the laptop, which just seems like a ridiculous amount for a machine that is primarily used for web browsing and some web development. Looking at the process list immediately confirmed what I already knew anyway: Chrome is basically hogging all of the memory. Out of 130 running processes, 74 - more than 50% - were Chrome alone.

 

looking at memory usage, it gets even worse: 


KiB Mem:   7864368 total,  7086624 used,   777744 free,    10616 buffers
KiB Swap:  8069116 total,    42912 used,  8026204 free.  1103676 cached Mem
 

In at least some cases, Chrome’s memory consumption is high enough that it will lead to swapping. If my understanding of Linux’s swap behavior is correct, it may be pathological in the specific case of something like Chrome. I imagine the following scenario:

  1. Chrome allocates lots of memory, and crosses the “available RAM” boundary.
  2. Kernel starts swapping stuff out.
  3. Chrome keeps allocating memory.
  4. Kernel keeps swapping, and hits things like X, the Cinnamon compositor, and/or the Chrome GPU process.
  5. At this point, the GUI is no longer responsive.

If Chrome keeps eating memory at a sufficient rate, this will only be cured when the limit of available swap is hit, at which point the OOM killer obliterates something. Which may or may not help, in this multi-process-browser world.

 

Often after extensive tab-switching in Chrome, or opening a new, likely “heavy” tab, the system hangs/stutters for about 5-10s in which the mouse might move slowly or not at all. When it comes back, free memory has drastically increased (I’ve seen levels of ~30-50% used afterwards) so I guess that a ton of stuff got swapped out.

 

Given I have already started to go down the rabbit hole, let’s take a look at why the hell Chrome is using all that memory. Now, I’ll admit that I do like my tabs - at least a few times a day I have to close some tabs because I can’t even see the icons on them any more. Typically I’m logged into two separate Chrome profiles, have about 2 windows per profile, and each window has anywhere from 10 to 50 tabs open.

 

But even so, some windows just use ridiculous amounts of memory. Gmail for example, stands at about 1.5GB, or almost 20%. I repeat: 20% of my laptop’s memory is used by a single Gmail tab. For giggles, I checked how much memory Outlook uses on my work Windows machine: 186 MB.

 

What’s even worse, Google makes it seem like it gives you the tools to actually look into where all the memory is going, but if you look closer, you’re in for a disappointment. In the developers console, there is a nice tab called “Memory”. It allows you to take allocation profiles and allocation timelines. So let’s look at what the allocation timeline looks like when you reload the Gmail tab:

 

Gmail in Chrome memory allocation timeline

 

After 45 seconds, Chrome claims the allocated memory is 195 MB - not that bad, and in the same ballpark as Outlook (even though: according to the network tracer, when reloading the Gmail tab, Chrome loaded 1.6MB over the network. It is still impressive to blow up this data to 12,000% of its original size).

 

Unfortunately, looking at the Chrome task manager tells a completely different story: 830+MB. So where the hell do the “missing” 600+MB memory go?!

 

Chrome task manager

 

At this point, of course my shiny new memory modules had already arrived, but I was in too deep to give up now. Pretty much the only place where this much memory could go was V8 garbage collection. Garbage collection is really a double edged sword - it gets rid of whole classes of bugs, including some with a security impact. But at the same time it means that web developers pretty much lose all their influence on the memory footprint of their application.

 

It becomes impossible for e.g. the Gmail developers to really do much about how much memory their tab uses (this is a simplification, and of course they could do things, but at a minimum it makes optimizations harder, in particular when standard frameworks are used rather than everything being written from scratch in plain ECMAScript).

 

Before we dive into the internal workings of the garbage collector, let’s take a look at how the heap itself is organized. Thankfully, Jay Conrod already did this in his excellent blog post here. Even though the post is originally from 2013, not much has changed since then in this regard.

V8 divides the heap into several different spaces:

  • New-space: Most objects are allocated here. New-space is small and is designed to be garbage collected very quickly, independent of other spaces.
  • Old-pointer-space: Contains most objects which may have pointers to other objects. Most objects are moved here after surviving in new-space for a while.
  • Old-data-space: Contains objects which just contain raw data (no pointers to other objects). Strings, boxed numbers, and arrays of unboxed doubles are moved here after surviving in new-space for a while.
  • Large-object-space: This space contains objects which are larger than the size limits of other spaces. Each object gets its own mmap’d region of memory. Large objects are never moved by the garbage collector.
  • Code-space: Code objects, which contain JITed instructions, are allocated here. This is the only space with executable memory (although Codes may be allocated in large-object-space, and those are executable, too).
  • Cell-space, property-cell-space and map-space: These spaces contain Cells, PropertyCells, and Maps, respectively. Each of these spaces contains objects which are all the same size and has some constraints on what kind of objects they point to, which simplifies collection.

With this in mind, let’s take a look at some of the latest memory “improvements” the V8 implemented in the last fall:

Memory usage difference between M53 and M55

 

Note how conspicuously, Gmail - other than the web search probably Google’s most used product, and likely one of the most used on the web in total - is missing from the list of tested applications. This may be because it behaves completely different with the changes implemented in M55:

 

Memory usage difference between M53 and M55

 

Two things are immediately obvious: (a) the memory use of Gmail dwarfs any of the other web sites the V8 developers chose to include in their stats. It’s hard not to suspect that this was done on purpose - after all Gmail is one of the applications that lots and lots of people have open all day long. But even more importantly, (b) the memory usage after the “optimizations” increased by more than 70%.

 

And Gmail does not appear to be the only web site where this is the case. A quick survey of the sites I use most showed that at least Stackoverflow had an almost as drastic increase in memory use, and several other a slightly smaller one (admittedly, most did use less memory though).

 

Reading through the changes that were made between M53 and M55, I was a bit stumped at what could have caused this. Then I noticed that Chrome introduced a special memory reduction mode which tunes several garbage collection heuristics to lower memory usage of the JavaScript garbage collected heap. It works like this:

  1. At the end of a full garbage collection, V8’s heap growing strategy determines when the next garbage collection will happen based on the amount of live objects with some additional slack. In memory reduction mode, V8 will use less slack resulting in less memory usage due to more frequent garbage collections.
  2. Moreover this estimate is treated as a hard limit, forcing unfinished incremental marking work to finalize in the main garbage collection pause. Normally, when not in memory reduction mode, unfinished incremental marking work may result in going over this limit arbitrarily to trigger the main garbage collection pause only when marking is finished.
  3. Memory fragmentation is further reduced by performing more aggressive memory compaction.

Folks familiar with heap management and garbage collection will immediately notice that this tweaks the inherent trade-off between garbage collection throughput, latency and memory consumption towards less memory consumption at the cost of higher latency and lower throughput. So, this should actually be good for memory consumption, right?

 

Unfortunately it turns out that under certain circumstances, e.g. for applications preferring arrays for storage rather than objects, this hugely increases the difference between the shallow size and the retained size of objects stored in memory - sometimes by as much as 100%. In the example of Gmail, the allocation timeline showed over 376,000 array objects, with a combined shallow size of 66MB, but a retained size of 109MB in M55, compared to 78MB in M53.

 

It’s kind of baffling to me that after more than 6 months that the V8 memory “optimizations” have been rolled out, this issue still has not been fixed - I verified it’s still present in M59. My best guess is that the optimizations work okay on mobile, due to a different memory reduction mode being used, and Google has stopped caring about their applications for the desktop.