Profiler fails to load large profiles in a .zip file

[animalan]

The profiler fails with Unserializing the profile failed: InternalError: allocation size overflow when trying to load a .zip file (~150MB) with a large (~900MB) profile (like this Windows Speedometer 3 Profile from Try).

What is interesting is if you unzip the .zip archive and upload the .json profile manually, it seems work.

When loading the .zip file with the profile:

When loading the profile manually after unzipping (successful):

┆Issue is synchronized with this Jira Task

[canova]

Hmm, interesting. I assume the size of the zip file + the uncompressed json is tripping the JS heap size limit. But there might be some weird/unnecessary copies that we can remove in that pipeline. It's worth investigating.

[canova]

@jensstutte shared another profile with us in the matrix channel from a SP3 profile, and it was a lot smaller compered to this one, but it still fails to open: profile link

ZIP size: 41.5MB
Unzipped size (correction, the file path has .json.gz, but it's actually a normal text file: 364MB

It seems like it's small enough and it should work. But I guess we do:

• Unzip it first
• decompress gzip (it turned out to be a text file even though it ended with .gz).
• Parse

But I think this was working before, so it makes me think that it's because of something we did recently...

[canova]

Huh, I managed to uncover the mystery.

I was looking into the recent changes, and one of them was #5959 which was the thing I made gzip inside zip files work.

I tried changing this line to:

diff --git a/src/actions/zipped-profiles.ts b/src/actions/zipped-profiles.ts
index acde01eee1..5353348dba 100644
--- a/src/actions/zipped-profiles.ts
+++ b/src/actions/zipped-profiles.ts
@@ -57,7 +57,7 @@
     try {
       // Attempt to unserialize the profile.
       const profile = await unserializeProfileOfArbitraryFormat(
-        await file.async('uint8array'),
+        await file.async('arraybuffer'),
         pathInZipFile
       );

So I tell the JSZip to return an ArrayBuffer instead of Uint8Array, and it actually worked. It looks like I can open both of the profiles with them. I was surprised to see that this worked because, ideally they should both be backed by the same ArrayBuffer, and Uint8Array should be a simple view wrapper around that, so I'm not entirely sure how this could make it work... I also at the source code of JSZip to see if there is anything silly there like an unnecessary copy, but I can't find it.

Then I decided to capture a profile and saw these:
Before: https://share.firefox.dev/4ufD0PH
After: https://share.firefox.dev/4tYwMUb

If you compare the two, there is a huge difference, the before one has a very long [TypedArray].prototype.join. I wasn't expecting that, so I used the JS source view to see which line it's happening, and found the culprit:

(can't share the JS sources yet, so putting as a screenshot)

It turns out, String(Uint8Array) actually stringifies the whole array into a string instead of just printing [object Uint8Array] like ArrayBuffer because it has its own toString in its prototype... Amazing.

Moral of the story, the profiler is great for debugging! 😄