A “managed” MediaSource is one where more control over the MediaSource and its associated objects has been given over to the User Agent.
IntroductionThe explicit goal of the Media Source Extensions specification is to transfer more control over the streaming of media data from the User Agent to the application running in the page. This transfer of control can and has added points of inefficiencies, where the page does not have the same level of capabilities, knowledge, or even goals as the User Agent.
Examples of these inefficiencies include the management of buffer levels, the timing and amount of network access, and media variant selection. These inefficiencies have largely been immaterial on relatively powerful devices like modern general purpose computers. However, on devices with narrower capabilities, it can be difficult to achieve the same quality of playback with the MediaSource API as is possible with native playback paths provided by the User Agent.
The goal of the ManagedMediaSource API is to transfer some control back to the User Agent from the application for the purpose of increasing playback efficiency and performance, while retaining the ability for pages to control streaming of media data.
GoalsA user loads a MSE-based website on a device with a limited amount of physical memory, and no ability to swap. The user plays some content on that website, and pauses that content. The system subsequently faces memory pressure, and requires applications (including the User Agent) to purge unused memory. If not enough memory is made available, those applications (including the User Agent) may be killed by the system in order to free up enough memory to perform the operation triggering this memory pressure.
The User Agent runs a version of the “Coded Frame Eviction” algorithm, removing ranges of buffered data in order to free memory for use by the system. At the end of this algorithm, the User Agent fires a “bufferedchange” event at every SourceBuffer affected by this algorithm, allowing the web application to be notified that it may need to re-request purged media data from the server before beginning playback.
Memory availability notificationWhen a call to appendBuffer is rejected with a QuotaExceededError exception, it can indicate the amount of excessive bytes or time that caused the error.
Network Streaming changesCurrently, a web application is allowed to append media data into a Source Buffer at any time, up until that Source Buffer’s “buffer full flag” is set, indicating no additional data is allowed to be appended. However, a constrained device may want to coalesce network use into a small window, and allow the network to query for battery and bandwidth reasons.
Alternatively, a device may have access to a high-speed network with high power use while the relevant communications interface is active (as can happen on 5G cellular). Using such a network may be beneficial in some circumstances:
To get these benefits without excessive battery drain, it's necessary to buffer more at once, and to limit streaming activity to specific windows so that the device's radio can be cycled on and off.
The User Agent would fire a “startstreaming” event at the MediaSource, indicating that the web application should begin streaming new media data. It would be up to the User Agent to determine when streaming should start, and could take current buffer levels, current time, network conditions, and other networking activity on the system.
When the User Agent determines that no further media streaming should take place, it would fire a “stopstreaming” event at the MediaSource, indicating to the web application that enough media data had been buffered to allow playback to continue successfully.
[Exposed=(Window,DedicatedWorker)]
interface ManagedMediaSource : MediaSource {
constructor();
readonly attribute boolean streaming;
attribute EventHandler onstartstreaming;
attribute EventHandler onendstreaming;
static boolean isTypeSupported (DOMString type);
};
[Exposed=Window,DedicatedWorker]
interface BufferedChangeEvent : Event {
constructor(DOMString type, BufferedChangeEventInit eventInitDict);
[SameObject] readonly attribute TimeRanges addedRanges;
[SameObject] readonly attribute TimeRanges removedRanges;
};
dictionary BufferedChangeEventInit : EventInit {
TimeRanges addedRanges;
TimeRanges removedRanges;
};
[Exposed=(Window,DedicatedWorker)]
interface ManagedSourceBuffer : SourceBuffer {
attribute EventHandler onbufferedchange;
};
async function setUpVideoStream(){
// Specific video format and codec
const mediaType = 'video/mp4; codecs="mp4a.40.2,avc1.4d4015"';
// Check if the type of video format / codect is supported.
if (!ManagedMediaSource.isTypeSupported(mediaType)) {
return; // Not supported, do something else.
}
// Set up video and its managed source.
const video = document.createElement("video");
const source = new ManagedMediaSource();
video.disableRemotePlayback = true;
video.controls = true;
await new Promise((resolve) => {
video.src = URL.createObjectURL(source);
source.addEventListener("sourceopen", resolve, { once: true });
document.body.appendChild(video);
});
const sourceBuffer = source.addSourceBuffer(mediaType);
// Set up the event handlers
sourceBuffer.onbufferedchange = (e) => {
console.log("onbufferedchange event fired.");
console.log(`Added Ranges: [${timeRangesToString(e.addedRanges)}]`);
console.log(`Removed Ranges: [${timeRangesToString(e.removedRanges)}]`);
};
source.onstartstreaming = async () => {
const response = await fetch("./videos/bipbop.mp4");
const buffer = await response.arrayBuffer();
await new Promise((resolve) => {
sourceBuffer.addEventListener("updateend", resolve, { once: true });
sourceBuffer.appendBuffer(buffer);
});
};
source.onendstreaming = async () => {
// Stop fetching new segments here
};
}
// Helper function...
function timeRangesToString(timeRanges) {
const ranges = [];
for (let i = 0; i < timeRanges?.length; i++) {
const range = [timeRanges.start(i), timeRanges.end(i)];
ranges.push(range);
}
return ranges.toString();
}
<body onload="setUpVideoStream()"></body>
TODO: discuss potential privacy protections if multiple origins try poke at this at the same time.
A concern is providing visibility to preferred quality if it is based on networking condition such as cellular or wifi etc.
To consider from MSE v2:
robwalch, johnBartos, dalecurtis, marcoscaceres, padenot and 16 morebriantbuttonRealAlphabet, o-t-w, dario-fiore and vahpetr
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4