Browser lied, performance died: a bit about HTML Canvas’ 2D context (and why you should read the docs)

Before I decided to revisit using WebGL for Ultrawidify’s aspect ratio detection, said autodetection was running off your standard, basic, non-webGL canvas context. This no longer happens, because CanvasRenderingContext2D is (was) slow. We are talking “double digit ms”, which resulted in some people experiencing dropped frames on higher resolution videos on certain hardware.

Which is why I was at least mildly conscious about performance implications of my decisions.

When Chrome’s dev tools helpfully suggested:

For those not in the know — if you want to extract video frame into something you can work with, you need a canvas and two functions.

1. drawImage() will draw current video frame to canvas
2. getImageData() will then convert the image on the canvas into something you can work with in code

Of the two functions, drawImage() is more expensive — and it’s not even close. Since drawImage() also resizes video frame to fit the canvas, it’s also necessary for this function to be hardware accelerated.

So when I went to MDN and saw this:

willReadFrequently

A boolean value that indicates whether or not a lot of read-back operations are planned. This will force the use of a software (instead of hardware accelerated) 2D canvas and can save memory when calling getImageData() frequently.

I was like: hell no.

Autodetection hasn’t had issues with memory leaks in literal years, and I haven’t had memory issues in literal years. There’s no universe in which willReadFrequently makes sense for my specific use case.

And I left it at that, until this week.

Look At This Noob Mistake

When you have an extension on Github, you’re gonna get your project forked. Very rarely is it gonna be someone who is interested in helping you out — more often it’s gonna be someone with a bone to pick.

These forks sometimes bring some fun stuff, such as: using docker to build the extension (which is hugely overkill). Sometimes, they contain things that you can yoink. And sometimes …

Or so I though, because this has gnawed on me for about a day now. Is using willReadFrequently really a noob mistake? It has just occurred me that I’ve always only assumed that using willReadFrequently is significantly slower because MDN said I lose hardware acceleration. But just because some page on the internet says it, it doesn’t mean it’s true. Does this guy know something I don’t? It’s more likely than you think.

Now that I have new graphs for debugging, I can actually … you know, test this. So turns out that no, I’m absolutely right once for a change.

So here’s three takeaways that I have from this.

1. I was right to avoid willReadFrequently flag. While it does make reading from canvas significantly faster (reading from canvas goes from ~3ms to ~1ms), that 2ms performance improvement costs us 35+ms performance penalty during the “draw to canvas” step of the process.
2. It appears that Chrome’s hardware acceleration of 2D canvas has improved massively in the last 6 months
3. Hot dang, hardware accelerated drawImage() on 2D canvas context is just as fast as doing it manually with WebGL. If I knew Chrome will make 2d canvas this much faster, I wouldn’t even bother with WebGL.

Though on the other hand, WebGL will still come in handy once I start porting manifest v3 version to Firefox.