I am trying to get a video to load from a stream. I would like to be able to "play" the video even if the entire video has not "downloaded" yet. The base64 data is ing in as a stream of buffer chunks.

const videoTag = document.getElementById('videoTag');
const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';
const mediaSource = new MediaSource();

videoTag.src = URL.createObjectURL(mediaSource);

await new Promise((resolve, reject) => {
    mediaSource.addEventListener('sourceopen', function (_) {
        resolve();
    });
});

const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);


for await (const chunk of ipfsNode.cat(CID)) {//Chunk is ing in as an array buffer
    sourceBuffer.appendBuffer(chunk);
}

sourceBuffer.addEventListener('updateend', async function (_) {
    mediaSource.endOfStream();
    $('#videoTag')[0].load();
});

I've tried using new MediaSource() and sourceBuffer.appendBuffer(chunk); to push new downloaded chunks to the video, but not only does the video not ever play, the method I am using requires that the entire video downloads before it can play.

How can I get my chunks of base64 data into a video?

EDIT (in response to ments): Regardless of whether or not I use a String or binary steam, I am a little confused on how I get a stream into a video.

I am trying to get a video to load from a stream. I would like to be able to "play" the video even if the entire video has not "downloaded" yet. The base64 data is ing in as a stream of buffer chunks.

const videoTag = document.getElementById('videoTag');
const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';
const mediaSource = new MediaSource();

videoTag.src = URL.createObjectURL(mediaSource);

await new Promise((resolve, reject) => {
    mediaSource.addEventListener('sourceopen', function (_) {
        resolve();
    });
});

const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);


for await (const chunk of ipfsNode.cat(CID)) {//Chunk is ing in as an array buffer
    sourceBuffer.appendBuffer(chunk);
}

sourceBuffer.addEventListener('updateend', async function (_) {
    mediaSource.endOfStream();
    $('#videoTag')[0].load();
});

I've tried using new MediaSource() and sourceBuffer.appendBuffer(chunk); to push new downloaded chunks to the video, but not only does the video not ever play, the method I am using requires that the entire video downloads before it can play.

How can I get my chunks of base64 data into a video?

EDIT (in response to ments): Regardless of whether or not I use a String or binary steam, I am a little confused on how I get a stream into a video.

• javascript
• video
• video-streaming
• html5-video
• ipfs

• Why base64? (And, where exactly is that ing from anyway...) – Brad Commented Nov 23, 2020 at 2:28
• It has to be either a base64 encoded string or a dataURL :/ And it is ing from my IPFS server that I own. – now_world Commented Nov 23, 2020 at 3:00
• Why do you believe it has to be base64/data URI? – Brad Commented Nov 23, 2020 at 3:04
• 7 Ok, so base64 is not what you want, nor strings. Keep everything in binary, otherwise you're adding 33% overhead in storage, memory, CPU, everywhere. – Brad Commented Nov 23, 2020 at 4:03
• 2 If your video file is available on your server as an mp4 then you just need to make sure your server supports range requests and that the mp4 has the 'MOOV' atom, the header, at the start and using the standard HTML5 tag will automatically request the video in 'chunks' and play it for you. If you want a more sophisticated streaming solution then you may want to use a dedicated streaming protocol like HLS or DASH - there are quite a few overviews available online and you can take a look at live examples on YouTube etc - see here: stackoverflow./a/42365034/334402 – Mick Commented Nov 26, 2020 at 10:22

2 Answers 2

You have two problems: 1. how to add arbitrary data to a video stream? 2. how to stream?

1. Given that you use a mp4 container (const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';) you can add arbitrary number of private streams, hence your encrypted stream (that can be base64 or bytes[] as well).

   From the MP4 registration authority the codecs encm (Encrypted/Protected metadata), encs (Encrypted Systems stream), enct (Encrypted Text) seems to be good candidates for your encrypted private stream.

2. ffmpeg have the ability to mux multiple streams into the final stream; this might be the solution on the backend side. Here there is no need to have the input/private streams finished in order to be able to mux and broadcast into the final stream; the reading should also happen without reaching the end of the stream.

Use a data: URL and directly insert that into the video src property.