I've recently been trying to generating video in the browser, and have thus been playing with two approaches:

• Using the whammy js library to bine webp frames into webm video. More details here.
• Using MediaRecorder and canvas.captureStream. More details here.

The whammy approach works well, but is only supported in Chrome, since it's the only browser that currently supports webp encoding (canvas.toDataURL("image/webp")). And so I'm using the captureStream approach as a backup for Firefox (and using libwebpjs for Safari).

So now on to my question: Is there a way to control the video quality of the canvas stream? And if not, has something like this been considered by the browsers / w3c?

Here's a screenshot of one of the frames of the video generated by whammy:

And here's the same frame generated by the MediaRecorder/canvas.captureStream approach:

My first thought is to artificially increase the resolution of the canvas that I'm streaming, but I don't want the output video to be bigger.

I've tried increasing the frame rate passed to the captureStream method (thinking that there may be some strange frame interpolation stuff happening), but this doesn't help. It actually degrades quality if I make it too high. My current theory is that the browser decides on the quality of the stream based on how much putational power it has access to. This makes sense, because if it's going to keep up with the frame rate that I've specified, then something has to give.

So the next thought is that I should slow down the rate at which I'm feeding the canvas with images, and then proportionally lower the FPS value that I pass into captureStream, but the problem with that is that even though I'd likely have solved the quality problem, I'd then end up with a video that runs slower than it's meant to.

Edit: Here's a rough sketch of the code that I'm using, in case it helps anyone in a similar situation.

I've recently been trying to generating video in the browser, and have thus been playing with two approaches:

• Using the whammy js library to bine webp frames into webm video. More details here.
• Using MediaRecorder and canvas.captureStream. More details here.

The whammy approach works well, but is only supported in Chrome, since it's the only browser that currently supports webp encoding (canvas.toDataURL("image/webp")). And so I'm using the captureStream approach as a backup for Firefox (and using libwebpjs for Safari).

So now on to my question: Is there a way to control the video quality of the canvas stream? And if not, has something like this been considered by the browsers / w3c?

Here's a screenshot of one of the frames of the video generated by whammy:

And here's the same frame generated by the MediaRecorder/canvas.captureStream approach:

My first thought is to artificially increase the resolution of the canvas that I'm streaming, but I don't want the output video to be bigger.

I've tried increasing the frame rate passed to the captureStream method (thinking that there may be some strange frame interpolation stuff happening), but this doesn't help. It actually degrades quality if I make it too high. My current theory is that the browser decides on the quality of the stream based on how much putational power it has access to. This makes sense, because if it's going to keep up with the frame rate that I've specified, then something has to give.

So the next thought is that I should slow down the rate at which I'm feeding the canvas with images, and then proportionally lower the FPS value that I pass into captureStream, but the problem with that is that even though I'd likely have solved the quality problem, I'd then end up with a video that runs slower than it's meant to.

Edit: Here's a rough sketch of the code that I'm using, in case it helps anyone in a similar situation.

• javascript
• canvas
• web-mediarecorder

• Can you share some of your code? – Helder Sepulveda Commented Sep 17, 2018 at 19:25
• 1 These are pression artifacts, and I'm afraid there no such options no... So first, these are not due to captureStream. You can check it by passing your stream to a videoElement's srcObject directly and you won't notice as much artifacts. The main problem is that most mon video codecs are not meant for displaying such graphics, think of it as what JPEG does. There is the videoBitsPerSecond option of the MediaRecorder but it will probably won't help here. (I tried and it didn't). On Chrome, I've got slightly less horrible results using vp8 codec than any other. – Kaiido Commented Sep 18, 2018 at 2:22
• @HelderSepu I've tried to strip away the irrelevant parts of my code, so this may have a bug or two: gist.github./josephrocca/ec073b3a90f936bec87bbd8e3e4c3486 – user993683 Commented Sep 18, 2018 at 3:39
• Just a note about your gist: if(MediaRecorder) should be if(window.MediaRecorder) otherwise you'll throw a ReferenceError, and instead of a 1ms setTimeout loop, you'd be better run a requestAnimationFrame one (I don't think you can record at more than 60FPS anyway) – Kaiido Commented Sep 18, 2018 at 5:30
• Ah, thanks for the MeidiaRecorder catch. The reason I went for setTimeout rather than requestAnimationFrame is because I figured that the latter is more prone to be throttled by the browser in case of lag - I'd prefer the user's browser to get laggy than to risk skipping frames. I also didn't set the timeout to 1/opts.fps because I figured that I should ideally update the canvas much faster than the captureStream frame rate or else I'd risk a sort of temporal aliasing effect. I might be mistaken there though. – user993683 Commented Sep 18, 2018 at 6:25

2 Answers 2

These are pression artifacts, and there is not much you can do for now...

Video codecs are built mainly with the idea of showing real-life colors and shapes, a bit like JPEG with a really low quality. They will also do their best to keep as less information as they can between keyframes (some using motion detection algorithm) so that they need less data to be stored.

These codecs normally have some configurable settings that will allow us to improve the constant-quality of the encoding, but MediaRecorder's specs being codec agnostic, they didn't provide (yet) an option in the API for us web-devs to set any other option than a fixed bit-rate (which won't help us more in here).

There is this proposal, which asks for such a feature though.

I saw some ridiculous behaviour of cpu and GPU on starting media recorder , like use so much resource but gives extremely low quality and hezzy webm video. I am using three js and trying to make a animation library like Manim in javascript. Due to low quality issue, i finally decided to remove those media recorder methods . Currently I using a method similar like manim.Now i have 4k export with High quality smooth Animation from Intel Pentium CPU G2010 (low end pc , even premier pro will be crash while getting load . Export there is far away)