Add support for querying concurrent/multiple playback capabilities #258

Motivation

Currently, navigator.mediaCapabilities.decodingInfo() allows developers to query if a device can smoothly and power-efficiently decode a single media configuration. However, modern web media experiences increasingly rely on concurrent video playback.

When developers build multiview features, gapless transitions, or interactive video, they have no standard way to determine if the client's hardware can handle multiple simultaneous decoders.

Real-world Use Cases

1. Multiview / Tiled Video: Sports broadcasting (e.g., 4 simultaneous 1080p live streams).
2. Gapless Playback / Cross-fading: Pre-buffering and decoding the "next episode" while the current episode's credits roll.
3. Interactive Media: Pre-decoding multiple branching video paths to ensure zero-latency switching.
4. Hardware DRM Limits: Devices often have strict TEE / Secure Memory limits. A smart TV might support one hardware DRM 4K stream, but fail if a second hardware-secure decoder is requested.

Proposed Solutions

To get us started, below are three potential approaches to address this, ordered from the simplest API surface to the most flexible. They are not polished and for discussion purpose only for now.

Option 1: Scalar Concurrency Property (Simplest)

Without modifying the input arguments, we add a single integer property to MediaCapabilitiesDecodingInfo. The application queries a single configuration, and the API returns how many simultaneous instances of that configuration the hardware can support.

Web API Change

dictionary MediaCapabilitiesDecodingInfo {
    required boolean supported;
    required boolean smooth;
    required boolean powerEfficient;
    // NEW: Maximum number of concurrent decodes supported for this configuration
    unsigned long maxConcurrent; 
};

Pros & Cons

• Pros: 100% backwards compatible. Extremely low barrier to entry for User Agents (UAs) to implement, as they only need to evaluate a single configuration against static hardware limits. This would work for some typical cases like multiview scenarios.
• Cons: Cannot explicitly evaluate heterogeneous streams (e.g., asking if the device can handle one 4K hardware DRM stream plus one 720p clear stream). Developers must infer capabilities for mixed streams.

Example

navigator.mediaCapabilities.decodingInfo(sportsStreamConfig).then(result => {
    if (result.supported && result.maxConcurrent >= 4) {
        enableQuadMultiviewUI();
    }
});

Option 2: Aggregate Context Query (Intermediate)

We introduce a new input dictionary and method that accepts an array of configurations, but returns a single MediaCapabilitiesDecodingInfo indicating whether the device can handle the entire bundle simultaneously.

Web API Change

dictionary MediaDecodingConcurrentConfiguration {
    required sequence<MediaDecodingConfiguration> streams;
};

// Returns a single boolean summary for the entire bundle
Promise<MediaCapabilitiesDecodingInfo> concurrentDecodingInfo(MediaDecodingConcurrentConfiguration configuration);

Pros & Cons

• Pros: Natively supports evaluating heterogeneous streams. Keeps the return logic simple for developers who just need a "yes/no" answer to enable a specific multi-stream feature.
• Cons: Lacks granularity. If the bundle is not supported, the developer does not know which stream in the sequence pushed the hardware over its limit.

Example

const bundle = { streams: [main4KConfig, pip720pConfig] };

navigator.mediaCapabilities.concurrentDecodingInfo(bundle).then(result => {
    if (result.supported && result.smooth) {
        startPictureInPicturePlayback();
    }
});

Option 3: Accumulative Evaluation (Most Complex/Flexible)

We introduce a method that accepts an array of configurations and returns a parallel array of results. Each result is evaluated in the context of the accumulated configs running concurrently.

Web API Change

// Returns a sequence of results mapping 1:1 with the input configurations
Promise<sequence<MediaCapabilitiesDecodingInfo>> multipleDecodingInfo(sequence<MediaDecodingConfiguration> configurations);

Pros & Cons

• Pros: The most flexible and comprehensive solution. Handles heterogeneous streams and provides precise, granular feedback on exactly what combinations the hardware can support.
• Cons: The largest API surface change. Feels like a disaster for UAs to implement.

Example

const configs = [main4KConfig, pip720pConfig];

navigator.mediaCapabilities.multipleDecodingInfo(configs).then(results => {
    const mainSupported = results[0].supported && results[0].smooth;
    const mainAndPipSupported = results[1].supported && results[1].smooth;
    ...
});

Open Questions for the Working Group

1. Is this a useful use case that we should provide a solution for?
2. Which of these three approaches strikes the right balance between developer needs and UA implementability? Or other better proposals?
3. How should the API account for the dynamic nature of system resources (e.g., evaluating concurrency limits when another browser tab or OS application is already utilizing the hardware decoder)?

Feedback from browser vendors and media developers is highly appreciated.