index.src.html

<pre class='metadata'>
Title: WebCodecs
Repository: w3c/webcodecs
Status: ED
ED: https://w3c.github.io/webcodecs/
TR: https://www.w3.org/TR/webcodecs/
Shortname: webcodecs
Level: None
Group: mediawg
Editor: Chris Cunningham, w3cid 114832, Google Inc. https://www.google.com/
Editor: Paul Adenot, w3cid 62410, Mozilla https://www.mozilla.org/
Editor: Bernard Aboba, w3cid 65611, Microsoft Corporation https://www.microsoft.com/

Abstract: This specification defines interfaces to codecs for encoding and
    decoding of audio, video, and images.

    This specification does not specify or require any particular codec or
    method of encoding or decoding. The purpose of this specification is to
    provide JavaScript interfaces to implementations of existing codec
    technology developed elsewhere. Implementers are free to
    support any combination of codecs or none at all.

Complain About:accidental-2119 yes
Markup Shorthands:css no, markdown yes, dfn yes
!Participate: <a href="https://github.com/w3c/webcodecs">Git Repository.</a>
!Participate: <a href="https://github.com/w3c/webcodecs/issues/new">File an issue.</a>
!Version History: <a href="https://github.com/w3c/webcodecs/commits">https://github.com/w3c/webcodecs/commits</a>
</pre>

<pre class=link-defaults>
spec:webidl; type:interface; text:Promise
spec:html; type:attribute; text:hidden
</pre>

<pre class='anchors'>
spec: media-source; urlPrefix: https://www.w3.org/TR/media-source/
    type: method
        for: MediaSource; text: isTypeSupported(); url: #dom-mediasource-istypesupported

spec: html; urlPrefix: https://html.spec.whatwg.org/multipage/;
    for: HTMLMediaElement;
        type: method; text: canPlayType(); url: #dom-navigator-canplaytype
    for: PlatformObject;
        type: attribute; text: [[Detached]]; url: structured-data.html#detached
    for: ImageBitmap;
        type: attribute; text: resizeWidth; url:#dom-imagebitmapoptions-resizewidth
        type: attribute; text: resizeHeight; url:#dom-imagebitmapoptions-resizeheight
        type: dfn; text: cropped to the source rectangle with formatting; url: imagebitmap-and-animations.html#cropped-to-the-source-rectangle-with-formatting
        type: dfn; text: bitmap data; url: imagebitmap-and-animations.html#concept-imagebitmap-bitmap-data
    for: Canvas;
        type: dfn; text: Check the usability of the image argument; url: canvas.html#check-the-usability-of-the-image-argument
    for: origin;
        type: dfn; text: origin; url: origin.html#concept-origin
    for: webappapis;
        type: dfn; text: global object; url: webappapis.html#global-object
        type: dfn; text: entry settings object; url: webappapis.html#entry-settings-object
    for: media;
        type: dfn; text: current playback position; url: media.html#current-playback-position
    type: dfn; text: live; url: infrastructure.html#live

spec: mediacapture-streams; urlPrefix: https://www.w3.org/TR/mediacapture-streams/
    for: mediaDevices;
        type: method; text: getUserMedia(); url: #dom-mediadevices-getusermedia

spec: mediacapture-screen-share; urlPrefix: https://w3c.github.io/mediacapture-screen-share/
    for: mediaDevices; type: method; text: getDisplayMedia(); url: #dom-mediadevices-getdisplaymedia

spec: mediacapture-main; urlPrefix: https://w3c.github.io/mediacapture-main/
    for:MediaStreamTrackState;
        type: enum-value; text: live; url: #idl-def-MediaStreamTrackState.live
        type: enum-value; text: ended; url: #idl-def-MediaStreamTrackState.ended

spec: mimesniff; urlPrefix: https://mimesniff.spec.whatwg.org/#
    type: dfn; text: MIME type; url: mime-type
    type: dfn; text: valid MIME type string; url:valid-mime-type

spec: infra; urlPrefix: https://infra.spec.whatwg.org/#
    type: dfn; text: queue; url: queues
    type: dfn; text: enqueuing; url: queue-enqueue;
    type: dfn; text: dequeued; url: queue-dequeue;
    type: dfn; text: empty; url: list-is-empty;
    type: dfn; text: list; url: lists;

spec: mediastream-recording; urlPrefix: https://www.w3.org/TR/mediastream-recording/#
    type: interface; text: MediaRecorder; url: mediarecorder
    type: interface; text: BitrateMode; url: bitratemode
    for: BitrateMode;
        type: enum-value; text: constant; url: dom-bitratemode-constant
        type: enum-value; text: variable; url: dom-bitratemode-variable

spec: media-capabilities; urlPrefix: https://w3c.github.io/media-capabilities/#
    type: method; text: decodingInfo(); url: dom-mediacapabilities-decodinginfo
    type: attribute; text: powerEfficient; url: dom-mediacapabilitiesinfo-powerefficient

spec: css-images-3; urlPrefix: https://www.w3.org/TR/css-images-3/
    type: dfn; text: natural dimensions; url: #natural-dimensions
    type: dfn; text: natural width; url: #natural-width
    type: dfn; text: natural height; url: #natural-height

spec: webrtc-svc; urlPrefix: https://w3c.github.io/webrtc-svc/
    type: dfn; text: scalability mode identifier; url:#scalabilitymodes*
</pre>

<pre class='biblio'>
{
  "H.273": {
    "href": "https://www.itu.int/rec/T-REC-H.273/en",
    "title": "Coding-independent code points for video signal type identification",
    "publisher": "ITU",
    "date": "December 2016"
  }
}
</pre>

<style>
main > dl > dd {
  margin-bottom: 1em;
}

table {
  width: 100%;
}

table#sample-types td, table#sample-types th {
  text-align: center;
}

table#sample-types .even {
    background-color: lightgrey;
}

.rfc2119 {
  text-transform: lowercase;
  font-variant: small-caps;
  font-style: normal;
}

</style>


Definitions {#definitions}
==========================

: <dfn>Codec</dfn>
:: Refers generically to an instance of AudioDecoder, AudioEncoder,
    VideoDecoder, or VideoEncoder.

: <dfn lt="Key Chunk|Key Frame">Key Chunk</dfn>
:: An encoded chunk that does not depend on any other frames for decoding. Also
    commonly referred to as a "key frame".

: <dfn>Internal Pending Output</dfn>
:: Codec outputs such as {{VideoFrame}}s that currently reside in the internal
    pipeline of the underlying codec implementation. The underlying codec
    implementation <em class="rfc2119">MAY</em> emit new outputs only when new
    inputs are provided. The underlying codec implementation <em
    class="rfc2119">MUST</em> emit all outputs in response to a flush.

: <dfn lt="system resources">Codec System Resources</dfn>
:: Resources including CPU memory, GPU memory, and exclusive handles to specific
    decoding/encoding hardware that <em class="rfc2119">MAY</em> be allocated by
    the User Agent as part of codec configuration or generation of {{AudioData}}
    and {{VideoFrame}} objects. Such resources <em class="rfc2119">MAY</em> be
    quickly exhausted and <em class="rfc2119">SHOULD</em> be released immediately
    when no longer in use.

: <dfn>Temporal Layer</dfn>
:: A grouping of {{EncodedVideoChunk}}s whose timestamp cadence produces a
    particular framerate. See {{VideoEncoderConfig/scalabilityMode}}.

: <dfn>Progressive Image</dfn>
:: An image that supports decoding to multiple levels of detail, with lower
    levels becoming available while the encoded data is not yet fully buffered.

: <dfn>Progressive Image Frame Generation</dfn>
:: A generational identifier for a given [=Progressive Image=] decoded output.
    Each successive generation adds additional detail to the decoded output.
    The mechanism for computing a frame's generation is implementer defined.

: <dfn>Primary Image Track</dfn>
:: An image track that is marked by the given image file as being the default
    track. The mechanism for indicating a primary track is format defined.

: <dfn>RGB Format</dfn>
:: A {{VideoPixelFormat}} containing red, green, and blue color channels in
    any order or layout (interleaved or planar), and irrespective of whether an
    alpha channel is present.

: <dfn>sRGB Color Space</dfn>
:: A {{VideoColorSpaceInit}} containing «[
    "primaries" → {{VideoColorPrimaries/bt709}},
    "transfer" → {{VideoTransferCharacteristics/iec61966-2-1}},
    "matrix" → {{VideoMatrixCoefficients/rgb}},
    "fullRange" → `true`]».

: <dfn>REC709 Color Space</dfn>
:: A {{VideoColorSpaceInit}} containing «[
    "primaries" → {{VideoColorPrimaries/bt709}},
    "transfer" → {{VideoTransferCharacteristics/bt709}},
    "matrix" → {{VideoMatrixCoefficients/bt709}},
    "fullRange" → `false`]».

<dfn>Codec Processing Model</dfn> {#codec-processing-model-section}
===================================================================

Background {#processing-model-background}
-----------------------------------------

<div class=non-normative>
  This section is non-normative.

  The codec interfaces defined by the specification are designed such that new
  codec tasks can be scheduled while previous tasks are still pending. For
  example, web authors can call `decode()` without waiting for a previous
  `decode()` to complete. This is achieved by offloading underlying codec tasks
  to a separate thread for parallel execution.

  This section describes threading behaviors as they are visible from the
  perspective of web authors. Implementers can choose to use more or less
  threads as long as the exernally visible behaviors of blocking and sequencing
  are maintained as follows.
</div>

Control Thread and Codec Thread {#control-thread-and-codec-thread}
------------------------------------------------------------------

All steps in this specification will run on either a [=control thread=] or
a [=codec thread=].

The <dfn>control thread</dfn> is the thread from which authors will construct
a [=codec=] and invoke its methods. Invoking a codec's methods will typically
result in the creation of [=control messages=] which are later executed on the
[=codec thread=]. Each [=webappapis/global object=] has a separate control
thread.

The <dfn>codec thread</dfn> is the thread from which a [=codec=] will
[=dequeue=] [=control messages=] and execute their steps. Each [=codec=]
instance has a separate codec thread. The lifetime of a codec thread matches
that of its associated [=codec=] instance.

The [=control thread=] uses a traditional event loop, as described in
[[!HTML]].

The [=codec thread=] uses a specialized [=codec processing loop=].

Communication from the [=control thread=] to the [=codec thread=] is done using
[=control message=] passing. Communication in the other direction is done using
regular event loop tasks.

Each [=codec=] instance has a single <dfn>control message queue</dfn> that is
a [=queue=] of <dfn>control messages</dfn>.

<dfn lt="Enqueues a control message|Queue a control message">Queuing a control
message</dfn> means [=enqueuing=] the message to a [=codec=]’s [=control
message queue=]. Invoking codec methods will often queue a control message
to schedule work.

<dfn lt="running a control message|control message steps">Running a control
message</dfn> means performing a sequence of steps specified by the method
that enqueued the message.

The <dfn>codec processing loop</dfn> <em class="rfc2119">MUST</em> run these steps:
1. While true:
    1. If the [=control message queue=] is empty, [=continue=].
    2. Dequeue |front message| from the [=control message queue=].
    3. Run [=control message steps=] described by |front message|.

AudioDecoder Interface {#audiodecoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface AudioDecoder {
  constructor(AudioDecoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long decodeQueueSize;

  undefined configure(AudioDecoderConfig config);
  undefined decode(EncodedAudioChunk chunk);
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<AudioDecoderSupport> isConfigSupported(AudioDecoderConfig config);
};

dictionary AudioDecoderInit {
  required AudioDataOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback AudioDataOutputCallback = undefined(AudioData output);
</xmp>

Internal Slots {#audiodecoder-internal-slots}
---------------------------------------------
: <dfn attribute for=AudioDecoder>[[codec implementation]]</dfn>
:: Underlying decoder implementation provided by the User Agent.
: <dfn attribute for=AudioDecoder>[[output callback]]</dfn>
:: Callback given at construction for decoded outputs.
: <dfn attribute for=AudioDecoder>[[error callback]]</dfn>
:: Callback given at construction for decode errors.
: <dfn attribute for=AudioDecoder>[[key chunk required]]</dfn>
:: A boolean indicating that the next chunk passed to {{AudioDecoder/decode()}}
    <em class="rfc2119">MUST</em> describe a [=key chunk=] as indicated by
    {{EncodedAudioChunk/[[type]]}}.
: <dfn attribute for=AudioDecoder>\[[state]]</dfn>
:: The current {{CodecState}} of this {{AudioDecoder}}.
: <dfn attribute for=AudioDecoder>\[[decodeQueueSize]]</dfn>
:: The number of pending decode requests. This number will decrease as the
    underlying codec is ready to accept new input.
: <dfn attribute for=AudioDecoder>[[pending flush promises]]</dfn>
:: A list of unresolved promises returned by calls to {{AudioDecoder/flush()}}.

Constructors {#audiodecoder-constructors}
-----------------------------------------
<dfn constructor for=AudioDecoder title="AudioDecoder(init)">
  AudioDecoder(init)
</dfn>
1. Let d be a new {{AudioDecoder}} object.
2. Assign init.output to {{AudioDecoder/[[output callback]]}}.
3. Assign init.error to {{AudioDecoder/[[error callback]]}}.
4. Assign `true` to {{AudioDecoder/[[key chunk required]]}}.
5. Assign `"unconfigured"` to {{AudioDecoder/[[state]]}}
6. Return d.

Attributes {#audiodecoder-attributes}
-------------------------------------
<dl>
  <dt>
    <dfn attribute for=AudioDecoder>state</dfn>
  </dt>
  <dd>Returns the value of {{AudioDecoder/[[state]]}}.</dd>
  <dt>
    <dfn attribute for=AudioDecoder>decodeQueueSize</dfn>
  </dt>
  <dd>
    Returns the value of {{AudioDecoder/[[decodeQueueSize]]}}.
  </dd>
</dl>

Methods {#audiodecoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=AudioDecoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the audio decoder for decoding
    chunks as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
        does not support |config|. Authors are encouraged to first check support
        by calling {{AudioDecoder/isConfigSupported()}} with |config|. User
        Agents don't have to support any particular codec type or configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid AudioDecoderConfig=], throw a
        {{TypeError}}.
    2. If {{AudioDecoder/[[state]]}} is `“closed”`, throw an {{InvalidStateError}}.
    3. Set {{AudioDecoder/[[state]]}} to `"configured"`.
    4. Set {{AudioDecoder/[[key chunk required]]}} to `true`.
    5. [=Queue a control message=] to configure the decoder with |config|.

    [=Running a control message=] to configure the decoder means running
    these steps:
    1. Let |supported| be the result of running the <a>Check Configuration
        Support</a> algorithm with |config|.
    2. If |supported| is `true`, assign
        {{AudioDecoder/[[codec implementation]]}} with an implementation
        supporting |config|.
    3. Otherwise, run the <a>Close AudioDecoder</a> algorithm with
        {{NotSupportedError}}.
  </dd>

  <dt><dfn method for=AudioDecoder>decode(chunk)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to decode the given |chunk|.

    When invoked, run these steps:
    1. If {{AudioDecoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    2. If {{AudioDecoder/[[key chunk required]]}} is `true`:
        1. If |chunk|.{{EncodedAudioChunk/[[type]]}} is not
            {{EncodedAudioChunkType/key}}, throw a {{DataError}}.
        2. Implementers <em class="rfc2119">SHOULD</em> inspect the |chunk|'s
            {{EncodedAudioChunk/[[internal data]]}} to verify that
            it is truly a [=key chunk=]. If a mismatch is detected, throw a
            {{DataError}}.
        3. Otherwise, assign `false` to
            {{AudioDecoder/[[key chunk required]]}}.
    3. Increment {{AudioDecoder/[[decodeQueueSize]]}}.
    4. [=Queue a control message=] to decode the |chunk|.

    [=Running a control message=] to decode the chunk means performing these
    steps:
    1. Attempt to use {{AudioDecoder/[[codec implementation]]}} to decode the
        chunk.
    2. If decoding results in an error, queue a task on the [=control thread=]
        event loop to run the [=Close AudioDecoder=] algorithm with
        {{EncodingError}}.
    3. Queue a task on the [=control thread=] event loop to decrement
        {{AudioDecoder/[[decodeQueueSize]]}}.
    4. Let |decoded outputs| be a [=list=] of decoded audio data outputs emitted
        by {{AudioDecoder/[[codec implementation]]}}.
    5. If |decoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output AudioData=] algorithm with
        |decoded outputs|.
  </dd>

  <dt><dfn method for=AudioDecoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{AudioDecoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Set {{AudioDecoder/[[key chunk required]]}} to `true`.
    3. Let |promise| be a new Promise.
    4. [=Queue a control message=] to flush the codec with |promise|.
    5. Append |promise| to {{AudioDecoder/[[pending flush promises]]}}.
    6. Return |promise|.

    [=Running a control message=] to flush the codec means performing these steps
        with |promise|.
    1. Signal {{AudioDecoder/[[codec implementation]]}} to emit all [=internal
        pending outputs=].
    2. Let |decoded outputs| be a [=list=] of decoded audio data outputs emitted
        by {{AudioDecoder/[[codec implementation]]}}.
    3. If |decoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output AudioData=] algorithm with
        |decoded outputs|.
    4. Queue a task on the [=control thread=] event loop to run these steps:
        1. Remove |promise| from {{AudioDecoder/[[pending flush promises]]}}.
        2. Resolve |promise|.
  </dd>

  <dt><dfn method for=AudioDecoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset AudioDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioDecoder>close()</df></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close AudioDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioDecoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{AudioDecoderSupport}} {{AudioDecoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{AudioDecoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid AudioDecoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |decoderSupport| be a newly constructed
            {{AudioDecoderSupport}}, initialized as follows:
            1. Set {{AudioDecoderSupport/config}} to the result of running the
                <a>Clone Configuration</a> algorithm with |config|.
            2. Set {{AudioDecoderSupport/supported}} to the result of running
                the <a>Check Configuration Support</a> algorithm with |config|.
        2. Resolve |p| with |decoderSupport|.
    5. Return  |p|.
  </dd>
</dl>

Algorithms {#audiodecoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn>Output AudioData</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |data| be an {{AudioData}}, initialized as follows:
            1. Assign `false` to {{platform object/[[Detached]]}}.
            2. Let |resource| be the [=media resource=] described by |output|.
            3. Let |resourceReference| be a reference to |resource|.
            4. Assign |resourceReference| to
                {{AudioData/[[resource reference]]}}.
            5. Let |timestamp| be the {{EncodedAudioChunk/[[timestamp]]}} of the
                {{EncodedAudioChunk}} associated with |output|.
            6. Assign |timestamp| to {{AudioData/[[timestamp]]}}.
            7. If |output| uses a recognized {{AudioSampleFormat}}, assign that
                format to {{AudioData/[[format]]}}. Otherwise, assign `null` to
                {{AudioData/[[format]]}}.
            8. Assign values to {{AudioData/[[sample rate]]}},
                {{AudioData/[[number of frames]]}}, and
                {{AudioData/[[number of channels]]}} as determined by |output|.
        3. Invoke {{AudioDecoder/[[output callback]]}} with |data|.
  </dd>
  <dt><dfn>Reset AudioDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{AudioDecoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{AudioDecoder/[[state]]}} to `"unconfigured"`.
    3. Signal {{AudioDecoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    4. Remove all [=control messages=] from the [=control message queue=].
    5. Set {{AudioDecoder/[[decodeQueueSize]]}} to zero.
    6. For each |promise| in {{AudioDecoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{AudioDecoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close AudioDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset AudioDecoder=] algorithm with |exception|.
    2. Set {{AudioDecoder/[[state]]}} to `"closed"`.
    3. Clear {{AudioDecoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}}, queue a task on
        the [=control thread=] event loop to invoke the {{AudioDecoder/[[error callback]]}} with |exception|.
  </dd>
</dl>

VideoDecoder Interface {#videodecoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface VideoDecoder {
  constructor(VideoDecoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long decodeQueueSize;

  undefined configure(VideoDecoderConfig config);
  undefined decode(EncodedVideoChunk chunk);
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<VideoDecoderSupport> isConfigSupported(VideoDecoderConfig config);
};

dictionary VideoDecoderInit {
  required VideoFrameOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback VideoFrameOutputCallback = undefined(VideoFrame output);
</xmp>

Internal Slots {#videodecoder-internal-slots}
---------------------------------------------
: <dfn attribute for=VideoDecoder>[[codec implementation]]</dfn>
:: Underlying decoder implementation provided by the User Agent.
: <dfn attribute for=VideoDecoder>[[output callback]]</dfn>
:: Callback given at construction for decoded outputs.
: <dfn attribute for=VideoDecoder>[[error callback]]</dfn>
:: Callback given at construction for decode errors.
: <dfn attribute for=VideoDecoder>[[active decoder config]]</dfn>
:: The {{VideoDecoderConfig}} that is actively applied.
: <dfn attribute for=VideoDecoder>[[key chunk required]]</dfn>
:: A boolean indicating that the next chunk passed to {{VideoDecoder/decode()}}
    <em class="rfc2119">MUST</em> describe a [=key chunk=] as indicated by
    {{EncodedVideoChunk/type}}.
: <dfn attribute for=VideoDecoder>\[[state]]</dfn>
:: The current {{CodecState}} of this {{VideoDecoder}}.
: <dfn attribute for=VideoDecoder>\[[decodeQueueSize]]</dfn>
:: The number of pending decode requests. This number will decrease as the
    underlying codec is ready to accept new input.
: <dfn attribute for=VideoDecoder>[[pending flush promises]]</dfn>
:: A list of unresolved promises returned by calls to {{VideoDecoder/flush()}}.


Constructors {#videodecoder-constructors}
-----------------------------------------
<dfn constructor for=VideoDecoder title="VideoDecoder(init)">
  VideoDecoder(init)
</dfn>
1. Let d be a new VideoDecoder object.
2. Assign `init.output` to the {{VideoDecoder/[[output callback]]}} internal slot.
3. Assign `init.error` to the {{VideoDecoder/[[error callback]]}} internal slot.
4. Assign `true` to {{VideoDecoder/[[key chunk required]]}}.
5. Assign `"unconfigured"` to {{VideoDecoder/[[state]]}}.
5. Return d.

Attributes {#videodecoder-attributes}
-------------------------------------
<dl>
  <dt>
    <dfn attribute for=VideoDecoder>state</dfn>
  </dt>
  <dd>
    Returns the value of {{VideoDecoder/[[state]]}}.
  </dd>
  <dt>
    <dfn attribute for=VideoDecoder>decodeQueueSize</dfn>
  </dt>
  <dd>
    Returns the value of {{VideoDecoder/[[decodeQueueSize]]}}.
  </dd>
</dl>

Methods {#videodecoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=VideoDecoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the video decoder for decoding
    chunks as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
      does not support |config|. Authors are encouraged to first check support by
      calling {{VideoDecoder/isConfigSupported()}} with |config|. User Agents
      don't have to support any particular codec type or configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid VideoDecoderConfig=], throw a
        {{TypeError}}.
    2. If {{VideoDecoder/[[state]]}} is `“closed”`, throw an {{InvalidStateError}}.
    3. Set {{VideoDecoder/[[state]]}} to `"configured"`.
    4. Set {{VideoDecoder/[[key chunk required]]}} to `true`.
    5. [=Queue a control message=] to configure the decoder with |config|.

    [=Running a control message=] to configure the decoder means running
    these steps:
    1. Let |supported| be the result of running the <a>Check Configuration
        Support</a> algorithm with |config|.
    2. If |supported| is `true`, assign
        {{VideoDecoder/[[codec implementation]]}} with an implementation
        supporting |config|.
    3. Otherwise, run the <a>Close VideoDecoder</a> algorithm with
        {{NotSupportedError}} and abort these steps.
    4. Set {{VideoDecoder/[[active decoder config]]}} to `config`.
  </dd>

  <dt><dfn method for=VideoDecoder>decode(chunk)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to decode the given |chunk|.

    NOTE: Authors are encouraged to call {{VideoFrame/close()}} on output
        {{VideoFrame}}s immediately when frames are no longer needed. The
        underlying [=media resource=]s are owned by the {{VideoDecoder}} and
        failing to release them (or waiting for garbage collection) can cause
        decoding to stall.

    NOTE: {{VideoDecoder}} requires that frames are output in the order they
        expect to be presented, commonly known as presentation order. When using
        some {{VideoDecoder/[[codec implementation]]}}s the User Agent will have to
        to reorder outputs into presentation order.

    When invoked, run these steps:
    1. If {{VideoDecoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    2. If {{VideoDecoder/[[key chunk required]]}} is `true`:
        1. If |chunk|.{{EncodedVideoChunk/type}} is not
            {{EncodedVideoChunkType/key}}, throw a {{DataError}}.
        2. Implementers <em class="rfc2119">SHOULD</em> inspect the |chunk|'s
            {{EncodedVideoChunk/[[internal data]]}} to verify that
            it is truly a [=key chunk=]. If a mismatch is detected, throw a
            {{DataError}}.
        3. Otherwise, assign `false` to
            {{VideoDecoder/[[key chunk required]]}}.
    3. Increment {{VideoDecoder/[[decodeQueueSize]]}}.
    4. [=Queue a control message=] to decode the |chunk|.

    [=Running a control message=] to decode the chunk means performing these steps:
    1. Attempt to use {{VideoDecoder/[[codec implementation]]}} to decode the
        chunk.
    2. If decoding results in an error, queue a task on the [=control thread=]
        event loop to run the [=Close VideoDecoder=] algorithm with
        {{EncodingError}}.
    3. Queue a task on the [=control thread=] event loop to decrement
        {{VideoDecoder/[[decodeQueueSize]]}}
    4. Let |decoded outputs| be a [=list=] of decoded video data outputs emitted
        by {{VideoDecoder/[[codec implementation]]}} in presentation order.
    5. If |decoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output VideoFrames=] algorithm with
        |decoded outputs|.
  </dd>

  <dt><dfn method for=VideoDecoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{VideoDecoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Set {{VideoDecoder/[[key chunk required]]}} to `true`.
    3. Let |promise| be a new Promise.
    4. [=Queue a control message=] to flush the codec with |promise|.
    5. Append |promise| to {{VideoDecoder/[[pending flush promises]]}}.
    6. Return |promise|.

    [=Running a control message=] to flush the codec means performing these steps
        with |promise|.
    1. Signal {{VideoDecoder/[[codec implementation]]}} to emit all [=internal
        pending outputs=].
    2. Let |decoded outputs| be a [=list=] of decoded video data outputs emitted
        by {{VideoDecoder/[[codec implementation]]}}.
    3. If |decoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output VideoFrames=] algorithm with
        |decoded outputs|.
    4. Queue a task on the [=control thread=] event loop to run these steps:
        1. Remove |promise| from {{VideoDecoder/[[pending flush promises]]}}.
        2. Resolve |promise|.
  </dd>

  <dt><dfn method for=VideoDecoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset VideoDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoDecoder>close()</df></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close VideoDecoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoDecoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{VideoDecoderSupport}} {{VideoDecoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{VideoDecoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid VideoDecoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |decoderSupport| be a newly constructed
            {{VideoDecoderSupport}}, initialized as follows:
            1. Set {{VideoDecoderSupport/config}} to the result of running the
                <a>Clone Configuration</a> algorithm with |config|.
            2. Set {{VideoDecoderSupport/supported}} to the result of running
                the <a>Check Configuration Support</a> algorithm with |config|.
        2. Resolve |p| with |decoderSupport|.
    5. Return  |p|.
  </dd>
</dl>

Algorithms {#videodecoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn>Output VideoFrames</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |timestamp| and |duration| be the
            {{EncodedVideoChunk/timestamp}} and {{EncodedVideoChunk/duration}}
            from the {{EncodedVideoChunk}} associated with |output|.
        2. Let |displayAspectWidth| and |displayAspectHeight| be undefined.
        3. If {{VideoDecoderConfig/displayAspectWidth}} and
            {{VideoDecoderConfig/displayAspectHeight}} [=map/exist=] in the
            {{VideoDecoder/[[active decoder config]]}}, assign their values to
            |displayAspectWidth| and |displayAspectHeight| respectively.
        4. Let |colorSpace| be the {{VideoColorSpace}} for |output| as detected
            by the codec implementation. If no {{VideoColorSpace}} is detected,
            let |colorSpace| be `undefined`.

            NOTE: The codec implementation can detect a {{VideoColorSpace}} by
                analyzing the bitstream. Detection is made on a best-effort
                basis. The exact method of detection is implementer defined and
                codec-specific.  Authors can override the detected
                {{VideoColorSpace}} by providing a
                {{VideoDecoderConfig/colorSpace}} in the {{VideoDecoderConfig}}.

        5. If {{VideoDecoderConfig/colorSpace}} [=map/exists=] in the
            {{VideoDecoder/[[active decoder config]]}}, assign its value to
            |colorSpace|.
        6. Let |frame| be the result of running the [=Create a VideoFrame=]
            algorithm with |output|, |timestamp|, |duration|,
            |displayAspectWidth|, |displayAspectHeight|, and |colorSpace|.
        7. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
  </dd>
  <dt><dfn>Reset VideoDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{VideoDecoder/state}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{VideoDecoder/state}} to `"unconfigured"`.
    3. Signal {{VideoDecoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    4. Remove all [=control messages=] from the [=control message queue=].
    5. Set {{VideoDecoder/[[decodeQueueSize]]}} to zero.
    6. For each |promise| in {{VideoDecoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{VideoDecoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close VideoDecoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset VideoDecoder=] algorithm with |exception|.
    2. Set {{VideoDecoder/state}} to `"closed"`.
    3. Clear {{VideoDecoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}}, queue a task on
        the [=control thread=] event loop to invoke the
        {{VideoDecoder/[[error callback]]}} with |exception|.
  </dd>
</dl>


AudioEncoder Interface {#audioencoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface AudioEncoder {
  constructor(AudioEncoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long encodeQueueSize;

  undefined configure(AudioEncoderConfig config);
  undefined encode(AudioData data);
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<AudioEncoderSupport> isConfigSupported(AudioEncoderConfig config);
};

dictionary AudioEncoderInit {
  required EncodedAudioChunkOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback EncodedAudioChunkOutputCallback =
    undefined (EncodedAudioChunk output,
               optional EncodedAudioChunkMetadata metadata = {});
</xmp>

Internal Slots {#audioencoder-internal-slots}
---------------------------------------------
<dl>
<dt><dfn attribute for=AudioEncoder>[[codec implementation]]</dfn></dt>
<dd>Underlying encoder implementation provided by the User Agent.</dd>
<dt><dfn attribute for=AudioEncoder>[[output callback]]</dfn></dt>
<dd>Callback given at construction for encoded outputs.</dd>
<dt><dfn attribute for=AudioEncoder>[[error callback]]</dfn></dt>
<dd>Callback given at construction for encode errors.</dd>
<dt><dfn attribute for=AudioEncoder>[[active encoder config]]</dfn></dt>
<dd>The {{AudioEncoderConfig}} that is actively applied.</dd>
<dt><dfn attribute for=AudioEncoder>[[active output config]]</dfn></dt>
<dd>
  The {{AudioDecoderConfig}} that describes how to decode the most recently
  emitted {{EncodedAudioChunk}}.
</dd>
<dt><dfn attribute for=AudioEncoder>\[[state]]</dfn></dt>
<dd>
  The current {{CodecState}} of this {{AudioEncoder}}.
</dd>
<dt><dfn attribute for=AudioEncoder>\[[encodeQueueSize]]</dfn></dt>
<dd>
  The number of pending encode requests. This number will decrease as the
  underlying codec is ready to accept new input.
</dd>
<dt><dfn attribute for=AudioEncoder>[[pending flush promises]]</dfn></dt>
<dd>
  A list of unresolved promises returned by calls to {{AudioEncoder/flush()}}.
</dd>
</dl>

Constructors {#audioencoder-constructors}
-----------------------------------------
<dfn constructor for=AudioEncoder title="AudioEncoder(init)">
  AudioEncoder(init)
</dfn>
1. Let e be a new AudioEncoder object.
2. Assign `init.output` to the {{AudioEncoder/[[output callback]]}} internal slot.
3. Assign `init.error` to the {{AudioEncoder/[[error callback]]}} internal slot.
4. Assign `"unconfigured"` to {{AudioEncoder/[[state]]}}.
5. Assign `null` to {{AudioEncoder/[[active encoder config]]}}.
6. Assign `null` to {{AudioEncoder/[[active output config]]}}.
7. Return e.

Attributes {#audioencoder-attributes}
-------------------------------------
<dl>
  <dt>
    <dfn attribute for=AudioEncoder>state</dfn>
  </dt>
  <dd>Returns the value of {{AudioEncoder/[[state]]}}.</dd>
  <dt>
    <dfn attribute for=AudioEncoder>encodeQueueSize</dfn>
  </dt>
  <dd>
    Returns the value of {{AudioEncoder/[[encodeQueueSize]]}}.
  </dd>
</dl>

Methods {#audioencoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=AudioEncoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the audio encoder for
    decoding chunks as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
        does not support |config|. Authors are encouraged to first check support
        by calling {{AudioEncoder/isConfigSupported()}} with |config|. User
        Agents don't have to support any particular codec type or configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid AudioEncoderConfig=], throw a
        {{TypeError}}.
    2. If {{AudioEncoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    3. Set {{AudioEncoder/[[state]]}} to `"configured"`.
    4. [=Queue a control message=] to configure the encoder using |config|.

    [=Running a control message=] to configure the encoder means performing these
    steps:
    1. Let |supported| be the result of running the <a>Check Configuration
        Support</a> algorithm with |config|.
    2. If |supported| is `true`, assign
        {{AudioEncoder/[[codec implementation]]}} with an implementation
        supporting |config|.
    3. Otherwise, run the <a>Close AudioEncoder</a> algorithm with
        {{NotSupportedError}} and abort these steps.
    4. Assign |config| to {{AudioEncoder/[[active encoder config]]}}
  </dd>

  <dt><dfn method for=AudioEncoder>encode(data)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to encode the given |data|.

    When invoked, run these steps:
    1. If the value of |data|'s {{platform object/[[Detached]]}} internal slot is
        `true`, throw a {{TypeError}}.
    2. If {{AudioEncoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    3. Let |dataClone| hold the result of running the [=Clone AudioData=]
        algorithm with |data|.
    4. Increment {{AudioEncoder/[[encodeQueueSize]]}}.
    5. [=Queue a control message=] to encode |dataClone|.

    [=Running a control message=] to encode the data means performing these steps.
    1. Attempt to use {{AudioEncoder/[[codec implementation]]}} to encode
        the [=media resource=] described by |dataClone|.
    2. If encoding results in an error, queue a task on the [=control thread=]
        event loop to run the [=Close AudioEncoder=] algorithm with
        {{EncodingError}}.
    3. Queue a task on the [=control thread=] event loop to decrement
        {{AudioEncoder/[[encodeQueueSize]]}}.
    4. Let |encoded outputs| be a [=list=] of encoded audio data outputs
        emitted by {{AudioEncoder/[[codec implementation]]}}.
    5. If |encoded outputs| is not empty, queue a task on the
        [=control thread=] event loop to run the [=Output EncodedAudioChunks=] algorithm with |encoded outputs|.
  </dd>

  <dt><dfn method for=AudioEncoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{AudioEncoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Let |promise| be a new Promise.
    3. [=Queue a control message=] to flush the codec with |promise|.
    4. Append |promise| to {{AudioEncoder/[[pending flush promises]]}}.
    5. Return |promise|.

    [=Running a control message=] to flush the codec means performing these steps
        with |promise|.
    1. Signal {{AudioEncoder/[[codec implementation]]}} to emit all [=internal
        pending outputs=].
    2. Let |encoded outputs| be a [=list=] of encoded audio data outputs
        emitted by {{AudioEncoder/[[codec implementation]]}}.
    3. If |encoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output EncodedAudioChunks=] algorithm with
        |encoded outputs|.
    4. Queue a task on the [=control thread=] event loop to run these steps:
        1. Remove |promise| from {{AudioEncoder/[[pending flush promises]]}}.
        2. Resolve |promise|.
  </dd>

  <dt><dfn method for=AudioEncoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset AudioEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioEncoder>close()</df></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close AudioEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=AudioEncoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{AudioEncoderSupport}} {{AudioEncoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{AudioEncoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid AudioEncoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |encoderSupport| be a newly constructed
            {{AudioEncoderSupport}}, initialized as follows:
            1. Set {{AudioEncoderSupport/config}} to the result of running the
                <a>Clone Configuration</a> algorithm with |config|.
            2. Set {{AudioEncoderSupport/supported}} to the result of running
                the <a>Check Configuration Support</a> algorithm with |config|.
        2. Resolve |p| with |encoderSupport|.
    5. Return  |p|.
  </dd>
</dl>

Algorithms {#audioencoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn>Output EncodedAudioChunks</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |chunkInit| be an {{EncodedAudioChunkInit}} with the following
            keys:
            1. Let {{EncodedAudioChunkInit/data}} contain the encoded audio data
                from |output|.
            2. Let {{EncodedAudioChunkInit/type}} be the
                {{EncodedAudioChunkType}} of |output|.
            3. Let {{EncodedAudioChunkInit/timestamp}} be the
                {{AudioData/timestamp}} from the AudioData associated with
                |output|.
            4. Let {{EncodedAudioChunkInit/duration}} be the
                {{AudioData/duration}} from the AudioData associated with
                |output|.
        2. Let |chunk| be a new {{EncodedAudioChunk}} constructed with
            |chunkInit|.
        3. Let |chunkMetadata| be a new {{EncodedAudioChunkMetadata}}.
        4. Let |encoderConfig| be the
            {{AudioEncoder/[[active encoder config]]}}.
        5. Let |outputConfig| be a new {{AudioDecoderConfig}} that describes
            |output|. Initialize |outputConfig| as follows:
            1. Assign |encoderConfig|.{{AudioEncoderConfig/codec}} to
                |outputConfig|.{{AudioDecoderConfig/codec}}.
            2. Assign |encoderConfig|.{{AudioEncoderConfig/sampleRate}} to
                |outputConfig|.{{AudioDecoderConfig/sampleRate}}.
            3. Assign to
                |encoderConfig|.{{AudioEncoderConfig/numberOfChannels}} to
                |outputConfig|.{{AudioDecoderConfig/numberOfChannels}}.
            4. Assign |outputConfig|.{{AudioDecoderConfig/description}} with a
                sequence of codec specific bytes as determined by the
                {{AudioEncoder/[[codec implementation]]}}. The User Agent <em
                class="rfc2119">MUST</em> ensure that the provided description
                could be used to correctly decode output.

                NOTE: The codec specific requirements for populating the
                    {{AudioDecoderConfig/description}} are described in the
                    [[WEBCODECS-CODEC-REGISTRY]].

        6. If |outputConfig| and {{AudioEncoder/[[active output config]]}} are
            not [=equal dictionaries=]:
            1. Assign |outputConfig| to
                |chunkMetadata|.{{EncodedAudioChunkMetadata/decoderConfig}}.
            2. Assign |outputConfig| to
                {{AudioEncoder/[[active output config]]}}.
        7. Invoke {{AudioEncoder/[[output callback]]}} with |chunk| and
            |chunkMetadata|.
  </dd>
  <dt><dfn>Reset AudioEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{AudioEncoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{AudioEncoder/[[state]]}} to `"unconfigured"`.
    3. Set {{AudioEncoder/[[active encoder config]]}} to `null`.
    4. Set {{AudioEncoder/[[active output config]]}} to `null`.
    5. Signal {{AudioEncoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    6. Remove all [=control messages=] from the [=control message queue=].
    7. Set {{AudioEncoder/[[encodeQueueSize]]}} to zero.
    8. For each |promise| in {{AudioEncoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{AudioEncoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close AudioEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset AudioEncoder=] algorithm with |exception|.
    2. Set {{AudioEncoder/[[state]]}} to `"closed"`.
    3. Clear {{AudioEncoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}}, queue a task on
        the [=control thread=] event loop to invoke the
        {{AudioDecoder/[[error callback]]}} with |exception|.
  </dd>
</dl>

EncodedAudioChunkMetadata {#encoded-audio-chunk-metadata}
---------------------------------------------------------
The following metadata dictionary is emitted by the
{{EncodedVideoChunkOutputCallback}} alongside an associated
{{EncodedVideoChunk}}.

<xmp class='idl'>
dictionary EncodedAudioChunkMetadata {
  AudioDecoderConfig decoderConfig;
};
</xmp>

: <dfn dict-member for=EncodedAudioChunkMetadata>decoderConfig</dfn>
:: A {{AudioDecoderConfig}} that authors <em class="rfc2119">MAY</em> use to
    decode the associated {{EncodedAudioChunk}}.


VideoEncoder Interface {#videoencoder-interface}
================================================

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface VideoEncoder {
  constructor(VideoEncoderInit init);

  readonly attribute CodecState state;
  readonly attribute unsigned long encodeQueueSize;

  undefined configure(VideoEncoderConfig config);
  undefined encode(VideoFrame frame, optional VideoEncoderEncodeOptions options = {});
  Promise<undefined> flush();
  undefined reset();
  undefined close();

  static Promise<VideoEncoderSupport> isConfigSupported(VideoEncoderConfig config);
};

dictionary VideoEncoderInit {
  required EncodedVideoChunkOutputCallback output;
  required WebCodecsErrorCallback error;
};

callback EncodedVideoChunkOutputCallback =
    undefined (EncodedVideoChunk chunk,
               optional EncodedVideoChunkMetadata metadata = {});
</xmp>

Internal Slots {#videoencoder-internal-slots}
---------------------------------------------
<dl>
<dt><dfn attribute for=VideoEncoder>[[codec implementation]]</dfn></dt>
<dd>Underlying encoder implementation provided by the User Agent.</dd>
<dt><dfn attribute for=VideoEncoder>[[output callback]]</dfn></dt>
<dd>Callback given at construction for encoded outputs.</dd>
<dt><dfn attribute for=VideoEncoder>[[error callback]]</dfn></dt>
<dd>Callback given at construction for encode errors.</dd>
<dt><dfn attribute for=VideoEncoder>[[active encoder config]]</dfn></dt>
<dd>The {{VideoEncoderConfig}} that is actively applied.</dd>
<dt><dfn attribute for=VideoEncoder>[[active output config]]</dfn></dt>
<dd>
  The {{VideoDecoderConfig}} that describes how to decode the most recently
  emitted {{EncodedVideoChunk}}.
</dd>
<dt><dfn attribute for=VideoEncoder>\[[state]]</dfn></dt>
<dd>
  The current {{CodecState}} of this {{VideoEncoder}}.
</dd>
<dt><dfn attribute for=VideoEncoder>\[[encodeQueueSize]]</dfn></dt>
<dd>
  The number of pending encode requests. This number will decrease as the
  underlying codec is ready to accept new input.
</dd>
<dt><dfn attribute for=VideoEncoder>[[pending flush promises]]</dfn></dt>
<dd>
  A list of unresolved promises returned by calls to {{VideoEncoder/flush()}}.
</dd>
</dl>

Constructors {#videoencoder-constructors}
-----------------------------------------
<dfn constructor for=VideoEncoder title="VideoEncoder(init)">
  VideoEncoder(init)
</dfn>
1. Let e be a new VideoEncoder object.
2. Assign `init.output` to the {{VideoEncoder/[[output callback]]}} internal slot.
3. Assign `init.error` to the {{VideoEncoder/[[error callback]]}} internal slot.
4. Assign "unconfigured" to {{VideoEncoder/[[state]]}}.
5. Return e.

Attributes {#videoencoder-attributes}
-------------------------------------
<dl>
  <dt>
    <dfn attribute for=VideoEncoder>state</dfn>
  </dt>
  <dd>Returns the value of {{VideoEncoder/[[state]]}}.</dd>
  <dt>
    <dfn attribute for=VideoEncoder>encodeQueueSize</dfn>
  </dt>
  <dd>
    Returns the value of {{VideoEncoder/[[encodeQueueSize]]}}.
  </dd>
</dl>

Methods {#videoencoder-methods}
-------------------------------
<dl>
  <dt><dfn method for=VideoEncoder>configure(config)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to configure the video encoder for
    decoding chunks as described by |config|.

    NOTE: This method will trigger a {{NotSupportedError}} if the User Agent
        does not support |config|. Authors are encouraged to first check support
        by calling {{VideoEncoder/isConfigSupported()}} with |config|. User
        Agents don't have to support any particular codec type or
        configuration.

    When invoked, run these steps:
    1. If |config| is not a [=valid VideoEncoderConfig=], throw a
        {{TypeError}}.
    2. If {{VideoEncoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    3. Set {{VideoEncoder/[[state]]}} to `"configured"`.
    4. [=Queue a control message=] to configure the encoder using |config|.

    [=Running a control message=] to configure the encoder means performing these
    steps:
    1. Let |supported| be the result of running the <a>Check Configuration
        Support</a> algorithm with |config|.
    2. If |supported| is `true`, assign
        {{VideoEncoder/[[codec implementation]]}} with an implementation
        supporting |config|.
    3. Otherwise, run the <a>Close VideoEncoder</a> algorithm with
        {{NotSupportedError}} and abort these steps.
    4. Assign |config| to {{VideoEncoder/[[active encoder config]]}}.
  </dd>

  <dt><dfn method for=VideoEncoder>encode(|frame|, |options|)</dfn></dt>
  <dd>
    [=Enqueues a control message=] to encode the given |frame|.

    When invoked, run these steps:
    1. If the value of |frame|'s {{platform object/[[Detached]]}} internal slot is
        `true`, throw a {{TypeError}}.
    2. If {{VideoEncoder/[[state]]}} is not `"configured"`, throw an
        {{InvalidStateError}}.
    3. Let |frameClone| hold the result of running the [=Clone VideoFrame=]
        algorithm with |frame|.
    4. Increment {{VideoEncoder/[[encodeQueueSize]]}}.
    5. [=Queue a control message=] to encode |frameClone|.

    [=Running a control message=] to encode the frame means performing these steps.
    1. Attempt to use {{VideoEncoder/[[codec implementation]]}} to encode
        |frameClone| according to |options|.
    2. If encoding results in an error, queue a task on the [=control thread=]
        event loop to run the [=Close VideoEncoder=] algorithm with
        {{EncodingError}}.
    3. Queue a task on the [=control thread=] event loop to decrement
        {{VideoEncoder/[[encodeQueueSize]]}}.
    4. Let |encoded outputs| be a [=list=] of encoded video data outputs
        emitted by {{VideoEncoder/[[codec implementation]]}}.
    5. If |encoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output EncodedVideoChunks=] algorithm with
        |encoded outputs|.
  </dd>

  <dt><dfn method for=VideoEncoder>flush()</dfn></dt>
  <dd>
    Completes all [=control messages=] in the [=control message queue=]
    and emits all outputs.

    When invoked, run these steps:
    1. If {{VideoEncoder/[[state]]}} is not `"configured"`, return
        [=a promise rejected with=] {{InvalidStateError}} {{DOMException}}.
    2. Let |promise| be a new Promise.
    3. [=Queue a control message=] to flush the codec with |promise|.
    4. Append |promise| to {{VideoEncoder/[[pending flush promises]]}}.
    5. Return |promise|.

    [=Running a control message=] to flush the codec means performing these steps
        with |promise|.
    1. Signal {{VideoEncoder/[[codec implementation]]}} to emit all [=internal
        pending outputs=].
    2. Let |encoded outputs| be a [=list=] of encoded video data outputs
        emitted by {{VideoEncoder/[[codec implementation]]}}.
    3. If |encoded outputs| is not empty, queue a task on the [=control thread=]
        event loop to run the [=Output EncodedVideoChunks=] algorithm with
        |encoded outputs|.
    4. Queue a task on the [=control thread=] event loop to run these steps:
        1. Remove |promise| from {{VideoEncoder/[[pending flush promises]]}}.
        2. Resolve |promise|.
  </dd>

  <dt><dfn method for=VideoEncoder>reset()</dfn></dt>
  <dd>
    Immediately resets all state including configuration,
    [=control messages=] in the [=control message queue=], and all pending
    callbacks.

    When invoked, run the [=Reset VideoEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoEncoder>close()</df></dt>
  <dd>
    Immediately aborts all pending work and releases [=system resources=].
    Close is final.

    When invoked, run the [=Close VideoEncoder=] algorithm with an
    {{AbortError}} {{DOMException}}.
  </dd>

  <dt><dfn method for=VideoEncoder>isConfigSupported(config)</dfn></dt>
  <dd>
    Returns a promise indicating whether the provided |config| is supported by
    the User Agent.

    NOTE: The returned {{VideoEncoderSupport}} {{VideoEncoderSupport/config}}
        will contain only the dictionary members that User Agent recognized.
        Unrecognized dictionary members will be ignored. Authors can detect
        unrecognized dictionary members by comparing
        {{VideoEncoderSupport/config}} to their provided |config|.

    When invoked, run these steps:
    1. If |config| is not a <a>valid VideoEncoderConfig</a>, return
        [=a promise rejected with=] {{TypeError}}.
    2. Let |p| be a new Promise.
    3. Let |checkSupportQueue| be the result of starting a new <a>parallel
        queue</a>.
    4. Enqueue the following steps to |checkSupportQueue|:
        1. Let |encoderSupport| be a newly constructed
            {{VideoEncoderSupport}}, initialized as follows:
            1. Set {{VideoEncoderSupport/config}} to the result of running the
                <a>Clone Configuration</a> algorithm with |config|.
            2. Set {{VideoEncoderSupport/supported}} to the result of running
                the <a>Check Configuration Support</a> algorithm with |config|.
        2. Resolve |p| with |encoderSupport|.
    5. Return  |p|.
  </dd>
</dl>

Algorithms {#videoencoder-algorithms}
-------------------------------------
<dl>
  <dt><dfn>Output EncodedVideoChunks</dfn> (with |outputs|)</dt>
  <dd>
    Run these steps:
    1. For each |output| in |outputs|:
        1. Let |chunkInit| be an {{EncodedVideoChunkInit}} with the following
            keys:
            1. Let {{EncodedVideoChunkInit/data}} contain the encoded video data
                from |output|.
            2. Let {{EncodedVideoChunkInit/type}} be the
                {{EncodedVideoChunkType}} of |output|.
            3. Let {{EncodedVideoChunkInit/timestamp}} be the
                {{VideoFrame/[[timestamp]]}} from the {{VideoFrame}}
                associated with |output|.
            4. Let {{EncodedVideoChunkInit/duration}} be the
                {{VideoFrame/[[duration]]}} from the {{VideoFrame}} associated
                with |output|.
        2. Let |chunk| be a new {{EncodedVideoChunk}} constructed with
            |chunkInit|.
        3. Let |chunkMetadata| be a new {{EncodedVideoChunkMetadata}}.
        4. Let |encoderConfig| be the
            {{VideoEncoder/[[active encoder config]]}}.
        5. Let |outputConfig| be a {{VideoDecoderConfig}} that describes
            |output|. Initialize |outputConfig| as follows:
            1. Assign `encoderConfig.codec` to `outputConfig.codec`.
            2. Assign `encoderConfig.width` to
                `outputConfig.visibleRect.width`.
            3. Assign `encoderConfig.height` to
                `outputConfig.visibleRect.height`.
            4. Assign `encoderConfig.displayWidth` to
                `outputConfig.displayWidth`.
            5. Assign `encoderConfig.displayHeight` to
                `outputConfig.displayHeight`.
            6. Assign the remaining keys of `outputConfig` as determined by
                {{VideoEncoder/[[codec implementation]]}}. The User Agent
                <em class="rfc2119">MUST</em> ensure that the configuration is
                completely described such that |outputConfig| could be used to
                correctly decode |output|.

                NOTE: The codec specific requirements for populating the
                    {{VideoDecoderConfig/description}} are described in the
                    [[WEBCODECS-CODEC-REGISTRY]].

        6. If |outputConfig| and {{VideoEncoder/[[active output config]]}} are
            not <a>equal dictionaries</a>:
            1. Assign |outputConfig| to
                |chunkMetadata|.{{EncodedVideoChunkMetadata/decoderConfig}}.
            2. Assign |outputConfig| to
                {{VideoEncoder/[[active output config]]}}.
        7. If |encoderConfig|.{{VideoEncoderConfig/scalabilityMode}}
            describes multiple [=temporal layers=]:
            1. Let |svc| be a new {{SvcOutputMetadata}} instance.
            2. Let |temporal_layer_id| be the zero-based index describing the
                temporal layer for |output|.
            3. Assign |temporal_layer_id| to
                |svc|.{{SvcOutputMetadata/temporalLayerId}}.
            4. Assign |svc| to
                |chunkMetadata|.{{EncodedVideoChunkMetadata/svc}}.
        8. If |encoderConfig|.{{VideoEncoderConfig/alpha}} is set to `"keep"`:
            1. Let |alphaSideData| be the encoded alpha data in |output|.
            2. Assign |alphaSideData| to
                |chunkMetadata|.{{EncodedVideoChunkMetadata/alphaSideData}}.
        9. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and
            |chunkMetadata|.
  </dd>
  <dt><dfn>Reset VideoEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. If {{VideoEncoder/[[state]]}} is `"closed"`, throw an {{InvalidStateError}}.
    2. Set {{VideoEncoder/[[state]]}} to `"unconfigured"`.
    3. Set {{VideoEncoder/[[active encoder config]]}} to `null`.
    4. Set {{VideoEncoder/[[active output config]]}} to `null`.
    5. Signal {{VideoEncoder/[[codec implementation]]}} to cease producing
        output for the previous configuration.
    6. Remove all [=control messages=] from the [=control message queue=].
    7. Set {{VideoEncoder/[[encodeQueueSize]]}} to zero.
    8. For each |promise| in {{VideoEncoder/[[pending flush promises]]}}:
        1. Reject |promise| with |exception|.
        2. Remove |promise| from {{VideoEncoder/[[pending flush promises]]}}.
  </dd>
  <dt><dfn>Close VideoEncoder</dfn> (with |exception|)</dt>
  <dd>
    Run these steps:
    1. Run the [=Reset VideoEncoder=] algorithm with |exception|.
    2. Set {{VideoEncoder/[[state]]}} to `"closed"`.
    3. Clear {{VideoEncoder/[[codec implementation]]}} and release associated
        [=system resources=].
    4. If |exception| is not an {{AbortError}} {{DOMException}}, queue a task on
        the [=control thread=] event loop to invoke the {{AudioDecoder/[[error callback]]}} with |exception|.
  </dd>
</dl>

EncodedVideoChunkMetadata {#encoded-video-chunk-metadata}
---------------------------------------------------------
The following metadata dictionary is emitted by the
{{EncodedVideoChunkOutputCallback}} alongside an associated
{{EncodedVideoChunk}}.

<xmp class='idl'>
dictionary EncodedVideoChunkMetadata {
  VideoDecoderConfig decoderConfig;
  SvcOutputMetadata svc;
  BufferSource alphaSideData;
};

dictionary SvcOutputMetadata {
  unsigned long temporalLayerId;
};
</xmp>

: <dfn dict-member for=EncodedVideoChunkMetadata>decoderConfig</dfn>
:: A {{VideoDecoderConfig}} that authors <em class="rfc2119">MAY</em> use to
    decode the associated {{EncodedVideoChunk}}.

: <dfn dict-member for=EncodedVideoChunkMetadata>svc</dfn>
:: A collection of metadata describing this {{EncodedVideoChunk}} with respect
    to the configured {{VideoEncoderConfig/scalabilityMode}}.

: <dfn dict-member for=EncodedVideoChunkMetadata>alphaSideData</dfn>
:: A {{BufferSource}} that contains the {{EncodedVideoChunk}}'s extra alpha
    channel data.

: <dfn dict-member for=SvcOutputMetadata>temporalLayerId</dfn>
:: A number that identifies the [=temporal layer=] for the associated
    {{EncodedVideoChunk}}.


Configurations{#configurations}
===============================

<dfn>Check Configuration Support</dfn> (with |config|) {#config-support}
------------------------------------------------------------------------
Run these steps:
1. If |config| is an {{AudioDecoderConfig}} or {{VideoDecoderConfig}} and the
    User Agent can't provide a <a>codec</a> that can decode the exact profile
    (where present), level (where present), and constraint bits (where present)
    indicated by the <a>codec string</a> in |config|.codec, return `false`.
2. If |config| is an {{AudioEncoderConfig}} or {{VideoEncoderConfig}}:
    1. If the <a>codec string</a> in |config|.codec contains a profile and the
        User Agent can't provide a <a>codec</a> that can encode the exact
        profile indicated by |config|.codec, return `false`.
    2. If the <a>codec string</a> in |config|.codec contains a level and the
        User Agent can't provide a <a>codec</a> that can encode to a level less
        than or equal to the level indicated by |config|.codec, return `false`.
    3. If the <a>codec string</a> in |config|.codec contains constraint bits and
        the User Agent can't provide a <a>codec</a> that can produce an encoded
        bitstream at least as constrained as indicated by |config|.codec, return
        `false`.
3. If the User Agent can provide a <a>codec</a> to support all entries of the
    |config|, including applicable default values for keys that are not
    included, return `true`.

    NOTE: The types {{AudioDecoderConfig}}, {{VideoDecoderConfig}},
        {{AudioEncoderConfig}}, and {{VideoEncoderConfig}} each define their
        respective configuration entries and defaults.

    NOTE: Support for a given configuration can change dynamically if the
        hardware is altered (e.g. external GPU unplugged) or if essential
        hardware resources are exhausted. User Agents describe support on
        a best-effort basis given the resources that are available at the time
        of the query.

2. Otherwise, return false.

<dfn>Clone Configuration</dfn> (with |config|) {#clone-config}
--------------------------------------------------------------

NOTE: This algorithm will copy only the dictionary members that the User Agent
    recognizes as part of the dictionary type.

Run these steps:
1. Let |dictType| be the type of dictionary |config|.
2. Let <var ignore=''>clone</var> be a new empty instance of |dictType|.
3. For each dictionary member |m| defined on |dictType|:
    1. If |m| does not [=map/exist=] in |config|, then [=iteration/continue=].
    2. If `config[m]` is a nested dictionary, set `clone[m]` to the result of
        recursively running the <a>Clone Configuration</a> algorithm with
        `config[m]`.
    3. Otherwise, assign the value of `config[m]` to `clone[m]`.


Signalling Configuration Support{#config-support-info}
------------------------------------------------------

### AudioDecoderSupport ### {#audio-decoder-support}
<xmp class='idl'>
dictionary AudioDecoderSupport {
  boolean supported;
  AudioDecoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=AudioDecoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{AudioDecoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=AudioDecoderSupport>config</dfn></dt>
  <dd>
    An {{AudioDecoderConfig}} used by the User Agent in determining the value of
    {{AudioDecoderSupport/supported}}.
  </dd>
</dl>

### VideoDecoderSupport ### {#video-decoder-support}
<xmp class='idl'>
dictionary VideoDecoderSupport {
  boolean supported;
  VideoDecoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=VideoDecoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{VideoDecoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=VideoDecoderSupport>config</dfn></dt>
  <dd>
    A {{VideoDecoderConfig}} used by the User Agent in determining the value of
    {{VideoDecoderSupport/supported}}.
  </dd>
</dl>

### AudioEncoderSupport ### {#audio-encoder-support}
<xmp class='idl'>
dictionary AudioEncoderSupport {
  boolean supported;
  AudioEncoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=AudioEncoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{AudioEncoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=AudioEncoderSupport>config</dfn></dt>
  <dd>
    An {{AudioEncoderConfig}} used by the User Agent in determining the value of
    {{AudioEncoderSupport/supported}}.
  </dd>
</dl>

### VideoEncoderSupport ### {#video-encoder-support}
<xmp class='idl'>
dictionary VideoEncoderSupport {
  boolean supported;
  VideoEncoderConfig config;
};
</xmp>

<dl>
  <dt><dfn dict-member for=VideoEncoderSupport>supported</dfn></dt>
  <dd>
    A boolean indicating the whether the corresponding
    {{VideoEncoderSupport/config}} is supported by the User Agent.
  </dd>
  <dt><dfn dict-member for=VideoEncoderSupport>config</dfn></dt>
  <dd>
    A {{VideoEncoderConfig}} used by the User Agent in determining the value of
    {{VideoEncoderSupport/supported}}.
  </dd>
</dl>

<dfn export>Codec String</dfn>{#config-codec-string}
----------------------------------------------------
A codec string describes a given codec format to be used for encoding or
decoding.

A <dfn>valid codec string</dfn> <em class="rfc2119">MUST</em> meet the following
conditions.
1. Is valid per the relevant codec specification (see examples below).
2. It describes a single codec.
3. It is unambiguous about codec profile, level, and constraint bits for codecs
    that define these concepts.

NOTE: In other media specifications, codec strings historically accompanied a
    [=MIME type=] as the "codecs=" parameter
    ({{MediaSource/isTypeSupported()}}, {{HTMLMediaElement/canPlayType()}})
    [[RFC6381]]. In this specification, encoded media is not containerized;
    hence, only the value of the codecs parameter is accepted.

NOTE: Encoders for codecs that define level and constraint bits have flexibility
    around these parameters, but won't produce bitstreams that have a higher
    level or are less constrained than requested.

The format and semantics for codec strings are defined by codec registrations
listed in the [[WEBCODECS-CODEC-REGISTRY]]. A compliant implementation <em
class="rfc2119">MAY</em> support any combination of codec registrations or
none at all.

AudioDecoderConfig{#audio-decoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary AudioDecoderConfig {
  required DOMString codec;
  required [EnforceRange] unsigned long sampleRate;
  required [EnforceRange] unsigned long numberOfChannels;
  BufferSource description;
};
</xmp>

To check if an {{AudioDecoderConfig}} is a <dfn export>valid AudioDecoderConfig</dfn>,
    run these steps:
1. If codec is not a <a>valid codec string</a>, return `false`.
2. Return `true`.

<dl>
  <dt><dfn dict-member for=AudioDecoderConfig>codec</dfn></dt>
  <dd>Contains a <a>codec string</a> in |config|.codec describing the codec.</dd>

  <dt><dfn dict-member for=AudioDecoderConfig>sampleRate</dfn></dt>
  <dd>The number of frame samples per second.</dd>

  <dt><dfn dict-member for=AudioDecoderConfig>numberOfChannels</dfn></dt>
  <dd>The number of audio channels.</dd>

  <dt><dfn dict-member for=AudioDecoderConfig>description</dfn></dt>
  <dd>
    A sequence of codec specific bytes, commonly known as extradata.

    NOTE: The registrations in the [[WEBCODECS-CODEC-REGISTRY]] describe whether/how to
        populate this sequence, corresponding to the provided
        {{AudioDecoderConfig/codec}}.
  </dd>
</dl>


VideoDecoderConfig{#video-decoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary VideoDecoderConfig {
  required DOMString codec;
  BufferSource description;
  [EnforceRange] unsigned long codedWidth;
  [EnforceRange] unsigned long codedHeight;
  [EnforceRange] unsigned long displayAspectWidth;
  [EnforceRange] unsigned long displayAspectHeight;
  VideoColorSpaceInit colorSpace;
  HardwareAcceleration hardwareAcceleration = "no-preference";
  boolean optimizeForLatency;
};
</xmp>

To check if a {{VideoDecoderConfig}} is a <dfn export>valid VideoDecoderConfig</dfn>,
run these steps:
1. If {{VideoDecoderConfig/codec}} is not a <a>valid codec string</a>, return
    `false`.
2. If one of {{VideoDecoderConfig/codedWidth}} or
    {{VideoDecoderConfig/codedHeight}} is provided but the other isn't,
    return `false`.
3. If {{VideoDecoderConfig/codedWidth}} = 0 or
    {{VideoDecoderConfig/codedHeight}} = 0, return `false`.
4. If one of {{VideoDecoderConfig/displayAspectWidth}} or
    {{VideoDecoderConfig/displayAspectHeight}} is provided but the other isn't,
    return `false`.
5. If {{VideoDecoderConfig/displayAspectWidth}} = 0 or
    {{VideoDecoderConfig/displayAspectHeight}} = 0, return `false`.
6. Return `true`.

<dl>
  <dt><dfn dict-member for=VideoDecoderConfig>codec</dfn></dt>
  <dd>Contains a codec string describing the codec.</dd>

  <dt><dfn dict-member for=VideoDecoderConfig>description</dfn></dt>
  <dd>
    A sequence of codec specific bytes, commonly known as extradata.

    NOTE: The registrations in the [[WEBCODECS-CODEC-REGISTRY]] describes
        whether/how to populate this sequence, corresponding to the provided
        {{VideoDecoderConfig/codec}}.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>codedWidth</dfn></dt>
  <dd>
    Width of the VideoFrame in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>codedHeight</dfn></dt>
  <dd>
    Height of the VideoFrame in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.
  </dd>

  NOTE: {{VideoDecoderConfig/codedWidth}} and {{VideoDecoderConfig/codedHeight}}
    are used when selecting a {{VideoDecoder/[[codec implementation]]}}.

  <dt><dfn dict-member for=VideoDecoderConfig>displayAspectWidth</dfn></dt>
  <dd>
    Horizontal dimension of the VideoFrame's aspect ratio when displayed.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>displayAspectHeight</dfn></dt>
  <dd>
    Vertical dimension of the VideoFrame's aspect ratio when displayed.
  </dd>

  NOTE: {{VideoFrame/displayWidth}} and {{VideoFrame/displayHeight}} can both be
      different from {{VideoDecoderConfig/displayAspectWidth}} and
      {{VideoDecoderConfig/displayAspectHeight}}, but have identical ratios, after scaling is applied when [=create a videoframe|creating the
      video frame=].

  <dt><dfn dict-member for=VideoDecoderConfig>colorSpace</dfn></dt>
  <dd>
    Configures the {{VideoFrame}}.{{VideoFrame/colorSpace}} for {{VideoFrame}}s
    associated with this {{VideoDecoderConfig}}. If
    {{VideoDecoderConfig/colorSpace}} [=map/exists=], the provided values will
    override any in-band values from the bitsream.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>hardwareAcceleration</dfn></dt>
  <dd>
    Hint that configures hardware acceleration for this codec. See
    {{HardwareAcceleration}}.
  </dd>

  <dt><dfn dict-member for=VideoDecoderConfig>optimizeForLatency</dfn></dt>
  <dd>
    Hint that the selected decoder SHOULD be configured to minimize the number
    of {{EncodedVideoChunk}}s that have to be decoded before a {{VideoFrame}}
    is output.

    NOTE: In addition to User Agent and hardware limitations, some codec
        bitstreams require a minimum number of inputs before any output can be
        produced.
  </dd>
</dl>


AudioEncoderConfig{#audio-encoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary AudioEncoderConfig {
  required DOMString codec;
  [EnforceRange] unsigned long sampleRate;
  [EnforceRange] unsigned long numberOfChannels;
  [EnforceRange] unsigned long long bitrate;
};
</xmp>

NOTE: Codec-specific extensions to {{AudioEncoderConfig}} are described in
    their registrations in the [[WEBCODECS-CODEC-REGISTRY]].

To check if an {{AudioEncoderConfig}} is a <dfn>valid AudioEncoderConfig</dfn>,
run these steps:
1. If {{AudioEncoderConfig/codec}} is not a <a>valid codec string</a>, return
    `false`.
2. Return `true`.

<dl>
  <dt><dfn dict-member for=AudioEncoderConfig>codec</dfn></dt>
  <dd>Contains a codec string describing the codec.</dd>

  <dt><dfn dict-member for=AudioEncoderConfig>sampleRate</dfn></dt>
  <dd>The number of frame samples per second.</dd>

  <dt><dfn dict-member for=AudioEncoderConfig>numberOfChannels</dfn></dt>
  <dd>The number of audio channels.</dd>

  <dt><dfn dict-member for=AudioEncoderConfig>bitrate</dfn></dt>
  <dd>
    The average bitrate of the encoded audio given in units of bits per second.
  </dd>
</dl>


VideoEncoderConfig{#video-encoder-config}
-----------------------------------------
<xmp class='idl'>
dictionary VideoEncoderConfig {
  required DOMString codec;
  required [EnforceRange] unsigned long width;
  required [EnforceRange] unsigned long height;
  [EnforceRange] unsigned long displayWidth;
  [EnforceRange] unsigned long displayHeight;
  [EnforceRange] unsigned long long bitrate;
  [EnforceRange] double framerate;
  HardwareAcceleration hardwareAcceleration = "no-preference";
  AlphaOption alpha = "discard";
  DOMString scalabilityMode;
  BitrateMode bitrateMode = "variable";
  LatencyMode latencyMode = "quality";
};
</xmp>

NOTE: Codec-specific extensions to {{VideoEncoderConfig}} are described in their
    registrations in the [[WEBCODECS-CODEC-REGISTRY]].

To check if a {{VideoEncoderConfig}} is a <dfn>valid VideoEncoderConfig</dfn>,
    run these steps:
1. If {{VideoEncoderConfig/codec}} is not a <a>valid codec string</a>, return
    `false`.
2. If {{VideoEncoderConfig/width}} = 0 or {{VideoEncoderConfig/height}}
    = 0, return `false`.
3. If {{VideoEncoderConfig/displayWidth}} = 0 or
    {{VideoEncoderConfig/displayHeight}} = 0, return `false`.
4. Return `true`.

<dl>
  <dt><dfn dict-member for=VideoEncoderConfig>codec</dfn></dt>
  <dd>Contains a <a>codec string</a> in |config|.codec describing the codec.</dd>

  <dt><dfn dict-member for=VideoEncoderConfig>width</dfn></dt>
  <dd>
    The encoded width of output {{EncodedVideoChunk}}s in pixels, prior to any
    display aspect ratio adjustments.

    The encoder <em class="rfc2119">MUST</em> scale any {{VideoFrame}} whose
    {{VideoFrame/[[visible width]]}} differs from this value.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>height</dfn></dt>
  <dd>
    The encoded height of output {{EncodedVideoChunk}}s in pixels, prior to any
    display aspect ratio adjustments.

    The encoder <em class="rfc2119">MUST</em> scale any {{VideoFrame}} whose
    {{VideoFrame/[[visible height]]}} differs from this value.
  </dd>
</dl>

<dl>
  <dt><dfn dict-member for=VideoEncoderConfig>displayWidth</dfn></dt>
  <dd>
    The intended display width of output {{EncodedVideoChunk}}s in pixels.
    Defaults to {{VideoEncoderConfig/width}} if not present.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>displayHeight</dfn></dt>
  <dd>
    The intended display height of output {{EncodedVideoChunk}}s in pixels.
    Defaults to {{VideoEncoderConfig/width}} if not present.
  </dd>
</dl>

<div class='note'>
  NOTE: Providing a {{VideoEncoderConfig/displayWidth}} or
      {{VideoEncoderConfig/displayHeight}} that differs from
      {{VideoEncoderConfig/width}} and {{VideoEncoderConfig/height}} signals
      that chunks are to be scaled after decoding to arrive at the final
      display aspect ratio.

      For many codecs this is merely pass-through information, but some codecs
      can sometimes include display sizing in the bitstream.
</div>

<dl>
  <dt><dfn dict-member for=VideoEncoderConfig>bitrate</dfn></dt>
  <dd>
    The average bitrate of the encoded video given in units of bits per second.
      
    NOTE: Authors are encouraged to additionally provide a
    {{VideoEncoderConfig/framerate}} to inform rate control.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>framerate</dfn></dt>
  <dd>
    The expected frame rate in frames per second, if known. This value, along
    with the frame {{VideoFrame/timestamp}}, <em class="rfc2119">SHOULD</em> be
    used by the video encoder to calculate the optimal byte length for each
    encoded frame. Additionally, the value <em class="rfc2119">SHOULD</em> be
    considered a target deadline for outputting encoding chunks when
    {{VideoEncoderConfig/latencyMode}} is set to {{LatencyMode/realtime}}.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>hardwareAcceleration</dfn></dt>
  <dd>
    Hint that configures hardware acceleration for this codec. See
    {{HardwareAcceleration}}.
  </dd>

<dt><dfn dict-member for=VideoEncoderConfig>alpha</dfn></dt>
  <dd>
    Whether the alpha component of the {{VideoFrame}} inputs <em
    class="rfc2119">SHOULD</em> be kept or discarded prior to encoding. If
    {{VideoEncoderConfig/alpha}} is equal to {{AlphaOption/discard}}, alpha data
    is always discarded, regardless of a {{VideoFrame}}'s
    {{VideoFrame/[[format]]}}.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>scalabilityMode</dfn></dt>
  <dd>
    An encoding [=scalability mode identifier=] as defined by [[WebRTC-SVC]].
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>bitrateMode</dfn></dt>
  <dd>
    Configures encoding to use a {{BitrateMode/constant}} or
    {{BitrateMode/variable}} bitrate as defined by [[MEDIASTREAM-RECORDING]].

    NOTE: The precise degree of bitrate fluctuation in either mode is
        implementation defined.
  </dd>

  <dt><dfn dict-member for=VideoEncoderConfig>latencyMode</dfn></dt>
  <dd>
    Configures latency related behaviors for this codec. See {{LatencyMode}}.
  </dd>

</dl>

Hardware Acceleration{#hardware-acceleration}
---------------------------------------------
<xmp class='idl'>
enum HardwareAcceleration {
  "no-preference",
  "prefer-hardware",
  "prefer-software",
};
</xmp>

When supported, hardware acceleration offloads encoding or decoding to
specialized hardware. {{HardwareAcceleration/prefer-hardware}} and
{{HardwareAcceleration/prefer-software}} are hints. While User Agents SHOULD
respect these values when possible, User Agents may ignore these values in some
or all circumstances for any reason.

To prevent fingerprinting, if a User Agent implements [[media-capabilities]],
the User Agent MUST ensure rejection or acceptance of a given
{{HardwareAcceleration}} preference reveals no additional information on top of
what is inherent to the User Agent and revealed by [[media-capabilities]]. If a
User Agent does not implement [[media-capabilities]] for reasons of
fingerprinting, they SHOULD ignore the {{HardwareAcceleration}} preference.

<div class='note'>
  NOTE: Good examples of when a User Agent can ignore
  {{HardwareAcceleration/prefer-hardware}} or
  {{HardwareAcceleration/prefer-software}} are for reasons of user privacy or
  circumstances where the User Agent determines an alternative setting would
  better serve the end user.

  Most authors will be best served by using the default of
  {{HardwareAcceleration/no-preference}}. This gives the User Agent flexibility
  to optimize based on its knowledge of the system and configuration. A common
  strategy will be to prioritize hardware acceleration at higher resolutions
  with a fallback to software codecs if hardware acceleration fails.

  Authors are encouraged to carefully weigh the tradeoffs when setting a
  hardware acceleration preference. The precise tradeoffs will be
  device-specific, but authors can generally expect the following:

  * Setting a value of {{HardwareAcceleration/prefer-hardware}} or
      {{HardwareAcceleration/prefer-software}} can significantly restrict what
      configurations are supported. It can occur that the user's device does not
      offer acceleration for any codec, or only for the most common profiles of
      older codecs. It can also occur that a given User Agent lacks a software
      based codec implementation.
  * Hardware acceleration does not simply imply faster encoding / decoding.
      Hardware acceleration often has higher startup latency but more consistent
      throughput performance. Acceleration will generally reduce CPU load.
  * For decoding, hardware acceleration is often less robust to inputs that are
      mislabeled or violate the relevant codec specification.
  * Hardware acceleration will often be more power efficient than purely
      software based codecs.
  * For lower resolution content, the overhead added by hardware acceleration
      can yield decreased performance and power efficiency compared to purely
      software based codecs.

  Given these tradeoffs, a good example of using "prefer-hardware" would be if
  an author intends to provide their own software based fallback via
  WebAssembly.

  Alternatively, a good example of using "prefer-software" would be if an author
  is especially sensitive to the higher startup latency or decreased robustness
  generally associated with hardware acceleration.
</div>

<dl>
  <dt><dfn enum-value for=HardwareAcceleration>no-preference</dfn></dt>
  <dd>
    Indicates that the User Agent <em class="rfc2119">MAY</em> use hardware
    acceleration if it is available and compatible with other aspects of the
    codec configuration.
  </dd>
  <dt><dfn enum-value for=HardwareAcceleration>prefer-software</dfn></dt>
  <dd>
    Indicates that the User Agent <em class="rfc2119">SHOULD</em> prefer a
    software codec implementation.  User Agents may ignore this value for any
    reason.

    NOTE: This can cause the configuration to be unsupported on platforms where
    an unaccelerated codec is unavailable or is incompatible with other aspects
    of the codec configuration.
  </dd>
  <dt><dfn enum-value for=HardwareAcceleration>prefer-hardware</dfn></dt>
  <dd>
    Indicates that the User Agent <em class="rfc2119">SHOULD</em> prefer
    hardware acceleration. User Agents may ignore this value for any reason.

    NOTE: This can cause the configuration to be unsupported on platforms where
    an accelerated codec is unavailable or is incompatible with other aspects of
    the codec configuration.
  </dd>
</dl>

Alpha Option{#alpha-option}
---------------------------
<xmp class='idl'>
enum AlphaOption {
  "keep",
  "discard",
};
</xmp>

Describes how the user agent <em class="rfc2119">SHOULD</em> behave when dealing
with alpha channels, for a variety of different operations.

<dl>
  <dt><dfn enum-value for=AlphaOption>keep</dfn></dt>
  <dd>
    Indicates that the user agent <em class="rfc2119">SHOULD</em> preserve alpha
    channel data for {{VideoFrame}}s, if it is present.
  </dd>
  <dt><dfn enum-value for=AlphaOption>discard</dfn></dt>
  <dd>
    Indicates that the user agent <em class="rfc2119">SHOULD</em> ignore or
    remove {{VideoFrame}}'s alpha channel data.
  </dd>
</dl>

Latency Mode{#latency-mode}
---------------------------
<xmp class='idl'>
enum LatencyMode {
  "quality",
  "realtime"
};
</xmp>

: <dfn enum-value for=LatencyMode>quality</dfn>
:: Indicates that the User Agent <em class="rfc2119">SHOULD</em> optimize for
    encoding quality. In this mode:
    * User Agents <em class="rfc2119">MAY</em> increase encoding latency to
        improve quality.
    * User Agents <em class="rfc2119">MUST</em> not drop frames to achieve the
        target {{VideoEncoderConfig/bitrate}} and/or
        {{VideoEncoderConfig/framerate}}.
    * {{VideoEncoderConfig/framerate}} <em class="rfc2119">SHOULD</em> not be
        used as a target deadline for emitting encoded chunks.

: <dfn enum-value for=LatencyMode>realtime</dfn>
:: Indicates that the User Agent <em class="rfc2119">SHOULD</em> optimize for
    low latency. In this
    mode:
    * User Agents <em class="rfc2119">MAY</em> sacrifice quality to improve
        latency.
    * User Agents <em class="rfc2119">MAY</em> drop frames to achieve the target
        {{VideoEncoderConfig/bitrate}} and/or {{VideoEncoderConfig/framerate}}.
    * {{VideoEncoderConfig/framerate}} <em class="rfc2119">SHOULD</em> be used
        as a target deadline for emitting encoded chunks.

Configuration Equivalence{#config-equivalence}
----------------------------------------------
Two dictionaries are <dfn>equal dictionaries</dfn> if they contain the same
keys and values. For nested dictionaries, apply this definition recursively.


VideoEncoderEncodeOptions{#video-encoder-options}
-------------------------------------------------

<xmp class='idl'>
dictionary VideoEncoderEncodeOptions {
  boolean keyFrame = false;
};
</xmp>

<dl>
  <dt><dfn dict-member for=VideoEncoderEncodeOptions>keyFrame</dfn></dt>
  <dd>
    A value of `true` indicates that the given frame MUST be encoded as a key
    frame. A value of `false` indicates that the User Agent has flexibility to
    decide whether the frame will be encoded as a [=key frame=].
  </dd>
</dl>


CodecState{#codec-state}
------------------------
<xmp class='idl'>
enum CodecState {
  "unconfigured",
  "configured",
  "closed"
};
</xmp>

<dl>
  <dt><dfn enum-value for=CodecState>unconfigured</dfn></dt>
  <dd>The codec is not configured for encoding or decoding.</dd>
  <dt><dfn enum-value for=CodecState>configured</dfn></dt>
  <dd>
    A valid configuration has been provided. The codec is ready for encoding or
        decoding.
  </dd>
  <dt><dfn enum-value for=CodecState>closed</dfn></dt>
  <dd>
    The codec is no longer usable and underlying [=system resources=] have
        been released.
  </dd>
</dl>

WebCodecsErrorCallback{#error-callback}
---------------------------------------
<xmp class='idl'>
callback WebCodecsErrorCallback = undefined(DOMException error);
</xmp>


Encoded Media Interfaces (Chunks) {#encoded-media-interfaces}
=============================================================
These interfaces represent chunks of encoded media.

EncodedAudioChunk Interface {#encodedaudiochunk-interface}
------------------------------------------------------------
<xmp class='idl'>
[Exposed=(Window,DedicatedWorker)]
interface EncodedAudioChunk {
  constructor(EncodedAudioChunkInit init);
  readonly attribute EncodedAudioChunkType type;
  readonly attribute long long timestamp;          // microseconds
  readonly attribute unsigned long long? duration; // microseconds
  readonly attribute unsigned long byteLength;

  undefined copyTo([AllowShared] BufferSource destination);
};

dictionary EncodedAudioChunkInit {
  required EncodedAudioChunkType type;
  required [EnforceRange] long long timestamp;    // microseconds
  [EnforceRange] unsigned long long duration;     // microseconds
  required BufferSource data;
};

enum EncodedAudioChunkType {
    "key",
    "delta",
};
</xmp>

### Internal Slots ### {#encodedaudiochunk-internal-slots}
: <dfn attribute for=EncodedAudioChunk>[[internal data]]</dfn></dt>
:: An array of bytes representing the encoded chunk data.
: <dfn attribute for=EncodedAudioChunk>\[[type]]</dfn>
:: Describes whether the chunk is a [=key chunk=].
: <dfn attribute for=EncodedAudioChunk>\[[timestamp]]</dfn>
:: The presentation timestamp, given in microseconds.
: <dfn attribute for=EncodedAudioChunk>\[[duration]]</dfn>
:: The presentation duration, given in microseconds.
: <dfn attribute for=EncodedAudioChunk>[[byte length]]</dfn>
:: The byte length of {{EncodedAudioChunk/[[internal data]]}}.

### Constructors ###{#encodedaudiochunk-constructors}
<dfn constructor for=EncodedAudioChunk title="EncodedAudioChunk(init)">
  EncodedAudioChunk(init)
</dfn>
1. Let |chunk| be a new {{EncodedAudioChunk}} object, initialized as follows
    1. Assign `init.type` to {{EncodedAudioChunk/[[type]]}}.
    2. Assign `init.timestamp` to {{EncodedAudioChunk/[[timestamp]]}}.
    3. If `init.duration` exists, assign it to
        {{EncodedAudioChunk/[[duration]]}}, or assign `null` otherwise.
    4. Assign a copy of `init.data` to {{EncodedAudioChunk/[[internal data]]}}.
    5. Assign `init.data.byteLength` to {{EncodedAudioChunk/[[byte length]]}};
5. Return |chunk|.

### Attributes ###{#encodedaudiochunk-attributes}
: <dfn attribute for=EncodedAudioChunk>type</dfn>
:: Returns the value of {{EncodedAudioChunk/[[type]]}}.

: <dfn attribute for=EncodedAudioChunk>timestamp</dfn>
:: Returns the value of {{EncodedAudioChunk/[[timestamp]]}}.

: <dfn attribute for=EncodedAudioChunk>duration</dfn>
:: Returns the value of {{EncodedAudioChunk/[[duration]]}}.

: <dfn attribute for=EncodedAudioChunk>byteLength</dfn>
:: Returns the value of {{EncodedAudioChunk/[[byte length]]}}.

### Methods ###{#encodedaudiochunk-methods}
: <dfn method for=EncodedAudioChunk>copyTo(destination)</dfn>
:: When invoked, run these steps:
    1. If the {{EncodedAudioChunk/[[byte length]]}} of this {{EncodedAudioChunk}} is
        greater than in |destination|, throw a {{TypeError}}.
    2. Copy the {{EncodedAudioChunk/[[internal data]]}} into |destination|.

EncodedVideoChunk Interface{#encodedvideochunk-interface}
-----------------------------------------------------------
<xmp class='idl'>
[Exposed=(Window,DedicatedWorker)]
interface EncodedVideoChunk {
  constructor(EncodedVideoChunkInit init);
  readonly attribute EncodedVideoChunkType type;
  readonly attribute long long timestamp;             // microseconds
  readonly attribute unsigned long long? duration;    // microseconds
  readonly attribute unsigned long byteLength;

  undefined copyTo([AllowShared] BufferSource destination);
};

dictionary EncodedVideoChunkInit {
  required EncodedVideoChunkType type;
  required [EnforceRange] long long timestamp;        // microseconds
  [EnforceRange] unsigned long long duration;         // microseconds
  required BufferSource data;
};

enum EncodedVideoChunkType {
    "key",
    "delta",
};
</xmp>

### Internal Slots ### {#encodedvideochunk-internal-slots}
: <dfn attribute for=EncodedVideoChunk>[[internal data]]</dfn></dt>
:: An array of bytes representing the encoded chunk data.
: <dfn attribute for=EncodedVideoChunk>\[[type]]</dfn>
:: The {{EncodedAudioChunkType}} of this {{EncodedVideoChunk}};
: <dfn attribute for=EncodedVideoChunk>\[[timestamp]]</dfn>
:: The presentation timestamp, given in microseconds.
: <dfn attribute for=EncodedVideoChunk>\[[duration]]</dfn>
:: The presentation duration, given in microseconds.
: <dfn attribute for=EncodedVideoChunk>[[byte length]]</dfn>
:: The byte length of {{EncodedVideoChunk/[[internal data]]}}.

### Constructors ###{#encodedvideochunk-constructors}
<dfn constructor for=EncodedVideoChunk title="EncodedVideoChunk(init)">
  EncodedVideoChunk(init)
</dfn>
1. Let |chunk| be a new {{EncodedVideoChunk}} object, initialized as follows
    1. Assign `init.type` to {{EncodedVideoChunk/[[type]]}}.
    2. Assign `init.timestamp` to {{EncodedVideoChunk/[[timestamp]]}}.
    3. If duration is present in init, assign `init.duration` to
        {{EncodedVideoChunk/[[duration]]}}. Otherwise, assign `null` to
        {{EncodedVideoChunk/[[duration]]}}.
    4. Assign a copy of `init.data` to {{EncodedVideoChunk/[[internal data]]}}.
    5. Assign `init.data.byteLength` to {{EncodedVideoChunk/[[byte length]]}};
3. Return |chunk|.

### Attributes ###{#encodedvideochunk-attributes}
: <dfn attribute for=EncodedVideoChunk>type</dfn>
:: Returns the value of {{EncodedVideoChunk/[[type]]}}.

: <dfn attribute for=EncodedVideoChunk>timestamp</dfn>
:: Returns the value of {{EncodedVideoChunk/[[timestamp]]}}.

: <dfn attribute for=EncodedVideoChunk>duration</dfn>
:: Returns the value of {{EncodedVideoChunk/[[duration]]}}.

: <dfn attribute for=EncodedVideoChunk>byteLength</dfn>
:: Returns the value of {{EncodedVideoChunk/[[byte length]]}}.

### Methods ###{#encodedvideochunk-methods}
: <dfn method for=EncodedVideoChunk>copyTo(destination)</dfn>
:: When invoked, run these steps:
    1. If {{EncodedVideoChunk/[[byte length]]}} is greater than
        the {{EncodedVideoChunk/[[byte length]]}} of |destination|, throw a
        {{TypeError}}.
    2. Copy the {{EncodedVideoChunk/[[internal data]]}} into |destination|.


Raw Media Interfaces {#raw-media-interfaces}
====================================================
These interfaces represent unencoded (raw) media.

Memory Model {#raw-media-memory-model}
--------------------------------------

### Background ### {#raw-media-memory-model-background}

This section is non-normative.

Decoded media data <em class="rfc2119">MAY</em> occupy a large amount of system
memory. To minimize the need for expensive copies, this specification defines a
scheme for reference counting (`clone()` and `close()`).

NOTE: Authors are encourage to call `close()` immediately when frames are
    no longer needed.

### Reference Counting ### {#raw-media-memory-model-reference-counting}

A <dfn>media resource</dfn> is storage for the actual pixel data or the audio
sample data described by a {{VideoFrame}} or {{AudioData}}.

The {{AudioData}} {{AudioData/[[resource reference]]}} and {{VideoFrame}}
{{VideoFrame/[[resource reference]]}} internal slots hold a reference to a
[=media resource=].

{{VideoFrame}}.{{VideoFrame/clone()}} and
{{AudioData}}.{{AudioData/clone()}} return new objects whose
`[[resource reference]]` points to the same [=media resource=] as the original
object.

{{VideoFrame}}.{{VideoFrame/close()}} and {{AudioData}}.{{AudioData/close()}}
will clear their [[resource reference]] slot, releasing the reference their
[=media resource=].

A [=media resource=] <em class="rfc2119">MUST</em> remain alive at least as long
as it continues to be referenced by a `[[resource reference]]`.

NOTE: When a [=media resource=] is no longer referenced by a
    `[[resource reference]]`, the resource can be destroyed. User Agents are
    encouraged to destroy such resources quickly to reduce memory pressure and
    facilitate resource reuse.

### Transfer and Serialization ### {#raw-media-serialization-and-transfer}

This section is non-normative.

{{AudioData}} and {{VideoFrame}} are both [=transferable objects|transferable=]
and [=serializable objects|serializable=] objects. Their transfer and
serialization steps are defined in [[#audiodata-transfer-serialization]] and
[[#videoframe-transfer-serialization]] respectively.

Transferring an {{AudioData}} or {{VideoFrame}} moves its `[[resource
reference]]` to the destination object and closes (as in {{AudioData/close()}})
the source object. Authors <em class="rfc2119">MAY</em> use this facility
to move an {{AudioData}} or {{VideoFrame}} between realms without copying the
underlying [=media resource=].

Serializing an {{AudioData}} or {{VideoFrame}} effectively clones (as in
{{VideoFrame/clone()}}) the source object, resulting in two objects that
reference the same [=media resource=]. Authors <em class="rfc2119">MAY</em> use
this facility to clone an {{AudioData}} or {{VideoFrame}} to another realm
without copying the underlying [=media resource=].

AudioData Interface {#audiodata-interface}
---------------------------------------------

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), Serializable, Transferable]
interface AudioData {
  constructor(AudioDataInit init);

  readonly attribute AudioSampleFormat? format;
  readonly attribute float sampleRate;
  readonly attribute unsigned long numberOfFrames;
  readonly attribute unsigned long numberOfChannels;
  readonly attribute unsigned long long duration;  // microseconds
  readonly attribute long long timestamp;          // microseconds

  unsigned long allocationSize(AudioDataCopyToOptions options);
  undefined copyTo([AllowShared] BufferSource destination, AudioDataCopyToOptions options);
  AudioData clone();
  undefined close();
};

dictionary AudioDataInit {
  required AudioSampleFormat format;
  required float sampleRate;
  required [EnforceRange] unsigned long numberOfFrames;
  required [EnforceRange] unsigned long numberOfChannels;
  required [EnforceRange] long long timestamp;  // microseconds
  required BufferSource data;
};
</xmp>

### Internal Slots ###{#audiodata-internal-slots}

: <dfn attribute for=AudioData>[[resource reference]]</dfn>
:: A reference to a [=media resource=] that stores the audio sample data for
    this {{AudioData}}.

: <dfn attribute for=AudioData>\[[format]]</dfn>
:: The {{AudioSampleFormat}} used by this {{AudioData}}. Will be `null` whenever
    the underlying format does not map to an {{AudioSampleFormat}} or when
    {{platform object/[[Detached]]}} is `true`.

: <dfn attribute for=AudioData>[[sample rate]]</dfn>
:: The sample-rate, in Hz, for this {{AudioData}}.

: <dfn attribute for=AudioData>[[number of frames]]</dfn>
:: The number of [=frames=] for this {{AudioData}}.

: <dfn attribute for=AudioData>[[number of channels]]</dfn>
:: The number of audio channels for this {{AudioData}}.

: <dfn attribute for=AudioData>\[[timestamp]]</dfn>
:: The presentation timestamp, in microseconds, for this {{AudioData}}.

### Constructors ###{#audiodata-constructors}
<dfn constructor for=AudioData title="AudioData(init)">
  AudioData(init)
</dfn>
1. If |init| is not a [=valid AudioDataInit=], throw a {{TypeError}}.
2. Let |frame| be a new {{AudioData}} object, initialized as follows:
    1. Assign `false` to {{platform object/[[Detached]]}}.
    2. Assign |init|.{{AudioDataInit/format}} to
        {{AudioData/[[format]]}}.
    3. Assign |init|.{{AudioDataInit/sampleRate}} to
        {{AudioData/[[sample rate]]}}.
    4. Assign |init|.{{AudioDataInit/numberOfFrames}} to
        {{AudioData/[[number of frames]]}}.
    5. Assign |init|.{{AudioDataInit/numberOfChannels}} to
        {{AudioData/[[number of channels]]}}.
    6. Assign |init|.{{AudioDataInit/timestamp}} to
        {{AudioData/[[timestamp]]}}.
    7. Let |resource| be a [=media resource=] containing a copy of
        |init|.{{AudioDataInit/data}}.
    8. Let |resourceReference| be a reference to |resource|.
    9. Assign |resourceReference| to {{AudioData/[[resource reference]]}}.
3. Return |frame|.

### Attributes ###{#audiodata-attributes}

: <dfn attribute for=AudioData>format</dfn>
:: The {{AudioSampleFormat}} used by this {{AudioData}}. Will be `null` whenever
    the underlying format does not map to a {{AudioSampleFormat}} or when
    {{platform object/[[Detached]]}} is `true`.

    The {{AudioData/format}} getter steps are to return
    {{AudioData/[[format]]}}.

: <dfn attribute for=AudioData>sampleRate</dfn>
:: The sample-rate, in Hz, for this {{AudioData}}.

    The {{AudioData/sampleRate}} getter steps are to return
    {{AudioData/[[sample rate]]}}.

: <dfn attribute for=AudioData>numberOfFrames</dfn>
:: The number of [=frames=] for this {{AudioData}}.

    The {{AudioData/numberOfFrames}} getter steps are to return
    {{AudioData/[[number of frames]]}}.

: <dfn attribute for=AudioData>numberOfChannels</dfn>
:: The number of audio channels for this {{AudioData}}.

    The {{AudioData/numberOfChannels}} getter steps are to return
    {{AudioData/[[number of channels]]}}.

: <dfn attribute for=AudioData>timestamp</dfn>
:: The presentation timestamp, in microseconds, for this {{AudioData}}.

    The {{AudioData/numberOfChannels}} getter steps are to return
    {{AudioData/[[timestamp]]}}.

: <dfn attribute for=AudioData>duration</dfn>
:: The duration, in microseconds, for this {{AudioData}}.

    The {{AudioData/duration}} getter steps are to:
    1. Let |microsecondsPerSecond| be `1,000,000`.
    2. Let |durationInSeconds| be the result of dividing
        {{AudioData/[[number of frames]]}} by {{AudioData/[[sample rate]]}}.
    3. Return the product of |durationInSeconds| and |microsecondsPerSecond|.

### Methods ###{#audiodata-methods}
: <dfn method for=AudioData>allocationSize(|options|)</dfn>
:: Returns the number of bytes required to hold the samples as described by
    |options|.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. Let |copyElementCount| be the result of running the
        [=Compute Copy Element Count=] algorithm with |options|.
    3. Let |destFormat| be the value of {{AudioData/[[format]]}}.
    4. If |options|.{{AudioDataCopyToOptions/format}} [=map/exists=], assign
        |options|.{{AudioDataCopyToOptions/format}} to |destFormat|.
    5. Let |bytesPerSample| be the number of bytes per sample, as defined by
        the |destFormat|.
    4. Return the product of multiplying |bytesPerSample| by
        |copyElementCount|.

: <dfn method for=AudioData>copyTo(|destination|, |options|)</dfn>
:: Copies the samples from the specified plane of the {{AudioData}} to the
    destination buffer.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. Let |copyElementCount| be the result of running the
        [=Compute Copy Element Count=] algorithm with |options|.
    3. Let |destFormat| be the value of {{AudioData/[[format]]}}.
    4. If |options|.{{AudioDataCopyToOptions/format}} [=map/exists=], assign
        |options|.{{AudioDataCopyToOptions/format}} to |destFormat|.
    5. Let |bytesPerSample| be the number of bytes per sample, as defined by
        the |destFormat|.
    6. If the product of multiplying |bytesPerSample| by |copyElementCount| is
        greater than `destination.byteLength`, throw a {{RangeError}}.
    7. Let |resource| be the [=media resource=] referenced by
        {{AudioData/[[resource reference]]}}.
    8. Let |planeFrames| be the region of |resource| corresponding to
        |options|.{{AudioDataCopyToOptions/planeIndex}}.
    9. Copy elements of |planeFrames| into |destination|, starting with the
        [=frame=] positioned at |options|.{{AudioDataCopyToOptions/frameOffset}}
        and stopping after |copyElementCount| samples have been copied. If
        |destFormat| does not equal {{AudioData/[[format]]}}, convert elements
        to the |destFormat| {{AudioSampleFormat}} while making the copy.

: <dfn method for=AudioData>clone()</dfn>
:: Creates a new AudioData with a reference to the same [=media resource=].

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. Return the result of running the [=Clone AudioData=] algorithm with
        [=this=].

: <dfn method for=AudioData>close()</dfn>
:: Clears all state and releases the reference to the [=media resource=].
    Close is final.

    When invoked, run the [=Close AudioData=] algorithm with [=this=].

### Algorithms ### {#audiodata-algorithms}

: <dfn>Compute Copy Element Count</dfn> (with |options|)
:: Run these steps:
    1. Let |destFormat| be the value of {{AudioData/[[format]]}}.
    2. If |options|.{{AudioDataCopyToOptions/format}} [=map/exists=], assign
        |options|.{{AudioDataCopyToOptions/format}} to |destFormat|.
    3. If |destFormat| describes an [=interleaved=] {{AudioSampleFormat}} and
        |options|.{{AudioDataCopyToOptions/planeIndex}} is greater than `0`,
        throw a {{RangeError}}.
    4. Otherwise, if |destFormat| describes a [=planar=] {{AudioSampleFormat}}
        and if |options|.{{AudioDataCopyToOptions/planeIndex}} is greater or
        equal to {{AudioData/[[number of channels]]}}, throw a
        {{RangeError}}.
    5. If {{AudioData/[[format]]}} does not equal |destFormat| and the User
        Agent does not support the requested {{AudioSampleFormat}} conversion,
        throw a {{NotSupportedError}} {{DOMException}}.
        Conversion to {{f32-planar}} <em class="rfc2119">MUST</em> always be
        supported.
    6. Let |frameCount| be the number of frames in the plane identified by
        |options|.{{AudioDataCopyToOptions/planeIndex}}.
    7. If |options|.{{AudioDataCopyToOptions/frameOffset}} is greater than or
        equal to |frameCount|, throw a {{RangeError}}.
    8. Let |copyFrameCount| be the difference of subtracting
        |options|.{{AudioDataCopyToOptions/frameOffset}} from |frameCount|.
    9. If |options|.{{AudioDataCopyToOptions/frameCount}} [=map/exists=]:
        1. If |options|.{{AudioDataCopyToOptions/frameCount}} is greater than
            |copyFrameCount|, throw a {{RangeError}}.
        2. Otherwise, assign |options|.{{AudioDataCopyToOptions/frameCount}}
            to |copyFrameCount|.
    10. Let |elementCount| be |copyFrameCount|.
    11. If |destFormat| describes an [=interleaved=]
        {{AudioSampleFormat}}, mutliply |elementCount| by
        {{AudioData/[[number of channels]]}}
    12. return |elementCount|.

: <dfn>Clone AudioData</dfn> (with |data|)
:: Run these steps:
    1. Let |clone| be a new {{AudioData}} initialized as follows:
        1. Let |resource| be the [=media resource=] referenced by |data|'s
            {{AudioData/[[resource reference]]}}.
        2. Let |reference| be a new reference to |resource|.
        3. Assign |reference| to {{AudioData/[[resource reference]]}}.
        4. Assign the values of |data|'s {{platform object/[[Detached]]}},
            {{AudioData/[[format]]}}, {{AudioData/[[sample rate]]}},
            {{AudioData/[[number of frames]]}},
            {{AudioData/[[number of channels]]}}, and
            {{AudioData/[[timestamp]]}} slots to the corresponding slots in
            |clone|.
    2. Return |clone|.


: <dfn>Close AudioData</dfn> (with |data|)
:: Run these steps:
    1. Assign `true` to |data|'s {{platform object/[[Detached]]}} internal slot.
    2. Assign `null` to |data|'s {{AudioData/[[resource reference]]}}.
    3. Assign `0` to |data|'s {{AudioData/[[sample rate]]}}.
    4. Assign `0` to |data|'s {{AudioData/[[number of frames]]}}.
    5. Assign `0` to |data|'s {{AudioData/[[number of channels]]}}.
    6. Assign `null` to |data|'s {{AudioData/[[format]]}}.

: To check if a {{AudioDataInit}} is a
    <dfn>valid AudioDataInit</dfn>, run these steps:
:: 1. If {{AudioDataInit/sampleRate}} less than or equal to `0`, return `false`.
    2. If {{AudioDataInit/numberOfFrames}} = `0`, return `false`.
    3. If {{AudioDataInit/numberOfChannels}} = `0`, return `false`.
    4. Verify {{AudioDataInit/data}} has enough data by running the following
        steps:
        1. Let |totalSamples| be the product of multiplying
            {{AudioDataInit/numberOfFrames}} by
            {{AudioDataInit/numberOfChannels}}.
        2. Let |bytesPerSample| be the number of bytes per sample, as defined by
            the {{AudioDataInit/format}}.
        3. Let |totalSize| be the product of multiplying |bytesPerSample| with
            |totalSamples|.
        4. Let |dataSize| be the size in bytes of {{AudioDataInit/data}}.
        5. If |dataSize| is less than |totalSize|, return false.
    5. Return `true`.

<div class='note'>
Note: It's expected that {{AudioDataInit}}'s {{AudioDataInit/data}}'s memory
    layout matches the expectations of the [=planar=] or [=interleaved=]
    {{AudioDataInit/format}}. There is no real way to verify whether the samples
    conform to their {{AudioSampleFormat}}.
</div>

### Transfer and Serialization ###{#audiodata-transfer-serialization}

: The {{AudioData}} [=transfer steps=] (with |value| and |dataHolder|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. For all {{AudioData}} internal slots in |value|, assign the value of
        each internal slot to a field in |dataHolder| with the same name as the
        internal slot.
    3. Run the [=Close AudioData=] algorithm with |value|.

: The {{AudioData}} [=transfer-receiving steps=] (with |dataHolder| and |value|)
    are:
:: 1. For all named fields in |dataHolder|, assign the value of each named
        field to the {{AudioData}} internal slot in |value| with the same name
        as the named field.

: The {{AudioData}} [=serialization steps=] (with |value|, |serialized|, and
    |forStorage|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. If |forStorage| is `true`, throw a {{TypeError}}.
    3. Let |resource| be the [=media resource=] referenced by
            |value|'s {{AudioData/[[resource reference]]}}.
    4. Let |newReference| be a new reference to |resource|.
    5. Assign |newReference| to |serialized|.[[resource reference]].
    6. For all remaining {{AudioData}} internal slots (excluding
        {{AudioData/[[resource reference]]}}) in |value|, assign the value of
        each internal slot to a field in |serialized| with the same name as the
        internal slot.

: The {{AudioData}} [=deserialization steps=] (with |serialized| and |value|)
    are:
:: 1. For all named fields in |serialized|, assign the value of each named
        field to the {{AudioData}} internal slot in |value| with the same name
        as the named field.

### AudioDataCopyToOptions ### {#audiodata-copy-to-options}

<xmp class='idl'>
dictionary AudioDataCopyToOptions {
  required [EnforceRange] unsigned long planeIndex;
  [EnforceRange] unsigned long frameOffset = 0;
  [EnforceRange] unsigned long frameCount;
  AudioSampleFormat format;
};
</xmp>

: <dfn dict-member for=AudioDataCopyToOptions>planeIndex</dfn>
:: The index identifying the plane to copy from.

: <dfn dict-member for=AudioDataCopyToOptions>frameOffset</dfn>
:: An offset into the source plane data indicating which [=frame=] to begin
    copying from. Defaults to `0`.

: <dfn dict-member for=AudioDataCopyToOptions>frameCount</dfn>
:: The number of [=frames=] to copy. If not provided, the copy will include all
    [=frames=] in the plane beginning with {{AudioDataCopyToOptions/frameOffset}}.

: <dfn dict-member for=AudioDataCopyToOptions>format</dfn>
:: The output {{AudioSampleFormat}} for the destination data. If not provided,
    the resulting copy will use [=this=] AudioData's {{AudioData/[[format]]}}.
    Invoking {{AudioData/copyTo()}} will throw a {{NotSupportedError}} if
    conversion to the requested format is not supported. Conversion from any
    {{AudioSampleFormat}} to {{f32-planar}} <em class="rfc2119">MUST</em> always
    be supported.

    NOTE: Authors seeking to integrate with [[WEBAUDIO]] can request
        {{f32-planar}} and use the resulting copy to create and {{AudioBuffer}}
        or render via {{AudioWorklet}}.

## Audio Sample Format ##{#audio-sample-formats}

An audio sample format describes the numeric type used to represent a
single sample (e.g. 32-bit floating point) and the arrangement of samples from
different channels as either [=interleaved=] or [=planar=]. The <dfn>audio
sample type</dfn> refers solely to the numeric type and interval used to store
the data, this is {{u8}}, {{s16}}, {{s32}}, or {{f32}} for respectively
unsigned 8-bits, signed 16-bits, signed 32-bits, and 32-bits
floating point number. The [[#audio-buffer-arrangement|audio buffer
arrangement]] refers solely to the way the samples are laid out in memory
([=planar=] or [=interleaved=]).

A <dfn>sample</dfn> refers to a single value that is the magnitude of a
signal at a particular point in time in a particular channel.

A <dfn>frame</dfn> or (sample-frame) refers to a set of values of all channels
of a multi-channel signal, that happen at the exact same time.

NOTE: Consequently if an audio signal is mono (has only one channel), a frame
and a sample refer to the same thing.

All audio [=samples=] in this specification are using linear pulse-code
modulation (Linear PCM): quantization levels are uniform between values.

NOTE: The Web Audio API, that is expected to be used with this specification,
also uses Linear PCM.

<xmp class='idl'>
enum AudioSampleFormat {
  "u8",
  "s16",
  "s32",
  "f32",
  "u8-planar",
  "s16-planar",
  "s32-planar",
  "f32-planar",
};
</xmp>

: <dfn enum-value for=AudioSampleFormat>u8</dfn>
:: [[WEBIDL#idl-octet|8-bit unsigned integer]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s16</dfn>
:: [[WEBIDL#idl-short|16-bit signed integer]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s32</dfn>
:: [[WEBIDL#idl-long|32-bit signed integer]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>f32</dfn>
:: [[WEBIDL#idl-float|32-bit float]] [=samples=] with [=interleaved=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>u8-planar</dfn>
:: [[WEBIDL#idl-octet|8-bit unsigned integer]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s16-planar</dfn>
:: [[WEBIDL#idl-short|16-bit signed integer]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>s32-planar</dfn>
:: [[WEBIDL#idl-long|32-bit signed integer]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

: <dfn enum-value for=AudioSampleFormat>f32-planar</dfn>
:: [[WEBIDL#idl-float|32-bit float]] [=samples=] with [=planar=] [[#audio-buffer-arrangement|channel arrangement]].

### Arrangement of audio buffer ### {#audio-buffer-arrangement}

When an {{AudioData}} has an {{AudioSampleFormat}} that is
<dfn>interleaved</dfn>, the audio samples from different channels are laid out
consecutively in the same buffer, in the order described in the section
[[#audio-channel-ordering]]. The {{AudioData}} has a single plane, that contains a
number of elements therefore equal to {{AudioData/[[number of frames]]}} *
{{AudioData/[[number of channels]]}}.

When an {{AudioData}} has an {{AudioSampleFormat}} that is
<dfn>planar</dfn>, the audio samples from different channels are laid out
in different buffers, themselves arranged in an order described in the section
[[#audio-channel-ordering]]. The {{AudioData}} has a number of planes equal to the
{{AudioData}}'s {{AudioData/[[number of channels]]}}. Each plane contains
{{AudioData/[[number of frames]]}} elements.

NOTE: The [[WEBAUDIO|Web Audio API]] currently uses {{f32-planar}} exclusively.

<div class='note'>
NOTE: The following diagram exemplifies the memory layout of [=planar=] versus
    [=interleaved=] {{AudioSampleFormat}}s

<img alt="Graphical representation the memory layout of interleaved and planar
    formats" src="images/planar_interleaved.svg" width="735" height="455">
</div>

### Magnitude of the audio samples ### {#audio-samples-magnitude}

The <dfn>minimum value</dfn> and <dfn>maximum value</dfn> of an audio sample,
for a particular audio sample type, are the values below which
(respectively above which) audio clipping might occur. They are otherwise regular
types, that can hold values outside this interval during intermediate
processing.

The <dfn>bias value</dfn> for an audio sample type is the value that often
corresponds to the middle of the range (but often the range is not symmetrical).
An audio buffer comprised only of values equal to the [=bias value=] is silent.

<table id="sample-types">
<thead>
<tr class="header">
<th>[=Audio sample type|Sample type=]</th>
<th>IDL type</th>
<th>[=Minimum value=]</th>
<th>[=Bias value=]</th>
<th>[=Maximum value=]</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>{{u8}}</td>
<td>[[WEBIDL#idl-octet|octet]]</td>
<td>0</td>
<td>128</td>
<td>+255</td>
</tr>
<tr class="even">
<td>{{s16}}</td>
<td>[[WEBIDL#idl-short|short]]</td>
<td>-32768</td>
<td>0</td>
<td>+32767</td>
</tr>
<tr class="odd">
<td>{{s32}}</td>
<td>[[WEBIDL#idl-long|long]]</td>
<td>-2147483648</td>
<td>0</td>
<td>+2147483647</td>
</tr>
<tr class="even">
<td>{{f32}}</td>
<td>[[WEBIDL#idl-float|float]]</td>
<td>-1.0</td>
<td>0.0</td>
<td>+1.0</td>
</tr>
</tbody>
</table>

NOTE: There is no data type that can hold 24 bits of information conveniently,
but audio content using 24-bit samples is common, so 32-bits integers are
commonly used to hold 24-bit content.

{{AudioData}} containing 24-bit samples SHOULD store those samples in {{s32}} or
{{f32}}. When samples are stored in {{s32}}, each sample MUST be left-shifted by
`8` bits. By virtue of this process, samples outside of the valid 24-bit range
([-8388608, +8388607]) will be clipped. To avoid clipping and ensure lossless
transport, samples <em class="rfc2119">MAY</em> be converted to {{f32}}.

NOTE: While clipping is unavoidable in {{u8}}, {{s16}}, and {{s32}} samples due
to their storage types, implementations SHOULD take care not to clip internally
when handling {{f32}} samples.

### Audio channel ordering ### {#audio-channel-ordering}

When decoding, the ordering of the audio channels in the resulting {{AudioData}}
MUST be the same as what is present in the {{EncodedAudioChunk}}.

When encoding, the ordering of the audio channels in the resulting
{{EncodedAudioChunk}} MUST be the same as what is preset in the given
{{AudioData}};

In other terms, no channel reordering is performed when encoding and decoding.

NOTE: The container either implies or specifies the channel mapping: the
channel attributed to a particular channel index.


VideoFrame Interface {#videoframe-interface}
--------------------------------------------

NOTE: {{VideoFrame}} is a {{CanvasImageSource}}. A {{VideoFrame}} can be
    passed to any method accepting a {{CanvasImageSource}}, including
    {{CanvasDrawImage}}'s {{CanvasDrawImage/drawImage()}}.

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker), Serializable, Transferable]
interface VideoFrame {
  constructor(CanvasImageSource image, optional VideoFrameInit init = {});
  constructor([AllowShared] BufferSource data, VideoFrameBufferInit init);

  readonly attribute VideoPixelFormat? format;
  readonly attribute unsigned long codedWidth;
  readonly attribute unsigned long codedHeight;
  readonly attribute DOMRectReadOnly? codedRect;
  readonly attribute DOMRectReadOnly? visibleRect;
  readonly attribute unsigned long displayWidth;
  readonly attribute unsigned long displayHeight;
  readonly attribute unsigned long long? duration;  // microseconds
  readonly attribute long long? timestamp;          // microseconds
  readonly attribute VideoColorSpace colorSpace;

  unsigned long allocationSize(
      optional VideoFrameCopyToOptions options = {});
  Promise<sequence<PlaneLayout>> copyTo(
      [AllowShared] BufferSource destination,
      optional VideoFrameCopyToOptions options = {});
  VideoFrame clone();
  undefined close();
};

dictionary VideoFrameInit {
  unsigned long long duration;  // microseconds
  long long timestamp;          // microseconds
  AlphaOption alpha = "keep";

  // Default matches image. May be used to efficiently crop. Will trigger
  // new computation of displayWidth and displayHeight using image's pixel
  // aspect ratio unless an explicit displayWidth and displayHeight are given.
  DOMRectInit visibleRect;

  // Default matches image unless visibleRect is provided.
  [EnforceRange] unsigned long displayWidth;
  [EnforceRange] unsigned long displayHeight;
};

dictionary VideoFrameBufferInit {
  required VideoPixelFormat format;
  required [EnforceRange] unsigned long codedWidth;
  required [EnforceRange] unsigned long codedHeight;
  required [EnforceRange] long long timestamp;  // microseconds
  [EnforceRange] unsigned long long duration;  // microseconds

  // Default layout is tightly-packed.
  sequence<PlaneLayout> layout;

  // Default visible rect is coded size positioned at (0,0)
  DOMRectInit visibleRect;

  // Default display dimensions match visibleRect.
  [EnforceRange] unsigned long displayWidth;
  [EnforceRange] unsigned long displayHeight;

  VideoColorSpaceInit colorSpace;
};
</xmp>

### Internal Slots ###{#videoframe-internal-slots}

: <dfn attribute for=VideoFrame>[[resource reference]]</dfn>
:: A reference to the [=media resource=] that stores the pixel data for
    this frame.

: <dfn attribute for=VideoFrame>\[[format]]</dfn>
:: A {{VideoPixelFormat}} describing the pixel format of the {{VideoFrame}}.
    Will be `null` whenever the underlying format does not map to a
    {{VideoPixelFormat}} or when {{platform object/[[Detached]]}} is `true`.

: <dfn attribute for=VideoFrame>[[coded width]]</dfn>
:: Width of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

: <dfn attribute for=VideoFrame>[[coded height]]</dfn>
:: Height of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

: <dfn attribute for=VideoFrame>[[visible left]]</dfn>
:: The number of pixels defining the left offset of the visible rectangle.

: <dfn attribute for=VideoFrame>[[visible top]]</dfn>
:: The number of pixels defining the top offset of the visible rectangle.

: <dfn attribute for=VideoFrame>[[visible width]]</dfn>
:: The width of pixels to include in visible rectangle, starting from
    {{VideoFrame/[[visible left]]}}.

: <dfn attribute for=VideoFrame>[[visible height]]</dfn>
:: The height of pixels to include in visible rectangle, starting from
    {{VideoFrame/[[visible top]]}}.

: <dfn attribute for=VideoFrame>[[display width]]</dfn>
:: Width of the {{VideoFrame}} when displayed after applying aspect ratio
    adjustments.

: <dfn attribute for=VideoFrame>[[display height]]</dfn>
:: Height of the {{VideoFrame}} when displayed after applying aspect ratio
    adjustments.

: <dfn attribute for=VideoFrame>\[[duration]]</dfn>
:: The presentation duration, given in microseconds. The duration is copied
        from the {{EncodedVideoChunk}} corresponding to this {{VideoFrame}}.

: <dfn attribute for=VideoFrame>\[[timestamp]]</dfn>
::  The presentation timestamp, given in microseconds. The timestamp is copied
    from the {{EncodedVideoChunk}} corresponding to this {{VideoFrame}}.

: <dfn attribute for=VideoFrame>[[color space]]</dfn>
:: The {{VideoColorSpace}} associated with this frame.

### Constructors ###{#videoframe-constructors}

<dfn constructor for=VideoFrame title="VideoFrame(image, init)">
  VideoFrame(image, init)
</dfn>
1. [=Canvas/Check the usability of the image argument=]. If this throws an
    exception or returns <var ignore=''>bad</var>, then throw an
    {{InvalidStateError}} {{DOMException}}.
2. If the [=origin/origin=] of |image|'s image data is not [=same origin=]
    with the [=webappapis/entry settings object=]'s
    [=origin/origin=], then throw a {{SecurityError}}
    {{DOMException}}.
3. Let |frame| be a new {{VideoFrame}}.
5. Switch on |image|:
    - {{HTMLImageElement}}
    - {{SVGImageElement}}
        1. If {{VideoFrameInit/timestamp}} does not [=map/exist=] in
            |init|, throw a {{TypeError}}.
        2. If |image|'s media data has no [=natural dimensions=]
            (e.g., it's a vector graphic with no specified content size), then
            throw an {{InvalidStateError}} {{DOMException}}.
        3. Let |resource| be a new [=media resource=] containing a copy of
            |image|'s media data. If this is an animated image, |image|'s
            [=bitmap data=] <em class="rfc2119">MUST</em> only be taken from the
            default image of the animation (the one that the format defines is
            to be used when animation is not supported or is disabled), or, if
            there is no such image, the first frame of the animation.
        4. Let |width| and |height| be the [=natural width=] and
            [=natural height=] of |image|.
        5. Run the [=VideoFrame/Initialize Frame With Resource and Size=]
            algorithm with |init|, |frame|, |resource|, |width|, and |height|

    - {{HTMLVideoElement}}
        1. If |image|'s {{HTMLMediaElement/networkState}} attribute is
            {{HTMLMediaElement/NETWORK_EMPTY}}, then throw an
            {{InvalidStateError}} {{DOMException}}.
        2. Let |currentPlaybackFrame| be the {{VideoFrame}} at the [=current
            playback position=].
        3. Run the [=VideoFrame/Initialize Frame From Other Frame=] algorithm
            with |init|, |frame|, and |currentPlaybackFrame|.

    - {{HTMLCanvasElement}}
    - {{ImageBitmap}}
    - {{OffscreenCanvas}}
        1. If {{VideoFrameInit/timestamp}} does not [=map/exist=] in
            |init|, throw a {{TypeError}}.
        2. Let |resource| be a new [=media resource=] containing a copy of
            |image|'s [=bitmap data=].

            NOTE: Implementers are encouraged to avoid a deep copy by using
                reference counting where feasible.

        3. Let |width| be `image.width` and |height| be `image.height`.
        4. Run the [=VideoFrame/Initialize Frame With Resource and Size=]
            algorithm with |init|, |frame|, |resource|, |width|, and |height|.

    - {{VideoFrame}}
        1. Run the [=VideoFrame/Initialize Frame From Other Frame=] algorithm
            with |init|, |frame|, and |image|.

6. Return |frame|.


<dfn constructor for=VideoFrame title="VideoFrame(data, init)">
  VideoFrame(data, init)
</dfn>
1. If |init| is not a [=valid VideoFrameBufferInit=], throw a {{TypeError}}.
2. Let |defaultRect| be «[ "x:" → `0`, "y" → `0`,
    "width" → |init|.{{VideoFrameBufferInit/codedWidth}}, "height" →
        |init|.{{VideoFrameBufferInit/codedWidth}} ]».
3. Let |overrideRect| be `undefined`.
4. If |init|.{{VideoFrameBufferInit/visibleRect}} [=map/exists=], assign its
    value to |overrideRect|.
4. Let |parsedRect| be the result of running the [=VideoFrame/Parse Visible
        Rect=] algorithm with |defaultRect|, |overrideRect|,
        |init|.{{VideoFrameBufferInit/codedWidth}},
        |init|.{{VideoFrameBufferInit/codedHeight}}, and
        |init|.{{VideoFrameBufferInit/format}}.
5. If |parsedRect| is an exception, return |parsedRect|.
6. Let |optLayout| be `undefined`.
7. If |options|.{{VideoFrameBufferInit/layout}} [=map/exists=], assign its value
    to |optLayout|.
8. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
    Layout and Allocation Size=] algorithm with |parsedRect|,
    |init|.{{VideoFrameBufferInit/format}}, and |optLayout|.
9. If |combinedLayout| is an exception, throw |combinedLayout|.
10. If `data.byteLength` is less than |combinedLayout|’s
    [=combined buffer layout/allocationSize=], throw a {{TypeError}}.
11. Let |resource| be a new [=media resource=] containing a copy of |data|. Use
        {{VideoFrameBufferInit/visibleRect}} and {{VideoFrameBufferInit/layout}}
        to determine where in |data| the pixels for each plane reside.

        The User Agent <em class="rfc2119">MAY</em> choose to allocate
        |resource| with a larger coded size and plane strides to improve memory
        alignment. Increases will be reflected by {{VideoFrame/codedWidth}} and
        {{VideoFrame/codedHeight}}.  Additionally, the User Agent <em
        class="rfc2119">MAY</em> use {{VideoFrameBufferInit/visibleRect}} to
        copy only the visible rectangle.  It <em class="rfc2119">MAY</em> also
        reposition the visible rectangle within |resource|. The final position
        will be reflected by {{VideoFrame/visibleRect}}.

12. Let |resourceCodedWidth| be the coded width of |resource|.
13. Let |resourceCodedHeight| be the coded height of |resource|.
14. Let |resourceVisibleLeft| be the left offset for the visible rectangle of
    |resource|.
15. Let |resourceVisibleTop| be the top offset for the visible rectangle of
    |resource|.

    ISSUE: The spec <em class="rfc2119">SHOULD</em> provide definitions (and
      possibly diagrams) for coded size, visible rectangle, and display size.
      See [#166](https://github.com/w3c/webcodecs/issues/166).

16. Let |frame| be a new {{VideoFrame}} object initialized as follows:
    1. Assign |resourceCodedWidth|, |resourceCodedHeight|,
        |resourceVisibleLeft|, and |resourceVisibleTop| to
        {{VideoFrame/[[coded width]]}}, {{VideoFrame/[[coded height]]}},
        {{VideoFrame/[[visible left]]}}, and {{VideoFrame/[[visible top]]}}
        respectively.
    2. If |init|.{{VideoFrameBufferInit/visibleRect}} [=map/exists=]:
        1. Let |truncatedVisibleWidth| be the value of
            {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/width}} after
            truncating.
        1. Assign |truncatedVisibleWidth| to {{VideoFrame/[[visible width]]}}.
        3. Let |truncatedVisibleHeight| be the value of
            {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/height}} after
            truncating.
        2. Assign |truncatedVisibleHeight| to {{VideoFrame/[[visible height]]}}.
    3. Otherwise:
        1. Assign {{VideoFrame/[[coded width]]}} to
            {{VideoFrame/[[visible width]]}}.
        2. Assign {{VideoFrame/[[coded height]]}} to
            {{VideoFrame/[[visible height]]}}.
    4. If |init|.{{VideoFrameBufferInit/displayWidth}} [=map/exists=], assign
        it to {{VideoFrame/[[display width]]}}. Otherwise, assign
        {{VideoFrame/[[visible width]]}} to {{VideoFrame/[[display width]]}}.
    5. If |init|.{{VideoFrameBufferInit/displayHeight}} [=map/exists=], assign
        it to {{VideoFrame/[[display height]]}}. Otherwise, assign
        {{VideoFrame/[[visible height]]}} to {{VideoFrame/[[display height]]}}.
    6. Assign |init|'s {{VideoFrameBufferInit/timestamp}} and
        {{VideoFrameBufferInit/duration}} to {{VideoFrame/[[timestamp]]}} and
        {{VideoFrame/[[duration]]}} respectively.
    7. Let |colorSpace| be `undefined`.
    8. If |init|.{{VideoFrameBufferInit/colorSpace}} [=map/exists=], assign its
        value to |colorSpace|.
    9. Assign the result of running the [=VideoFrame/Pick Color Space=]
        algorithm, with |colorSpace| and {{VideoFrame/[[format]]}}, to
        {{VideoFrame/[[color space]]}}.
17. Return |frame|.

### Attributes ###{#videoframe-attributes}
: <dfn attribute for=VideoFrame>format</dfn>
:: Describes the arrangement of bytes in each plane as well as the number and
    order of the planes. Will be `null` whenever the underlying format does not
    map to a {{VideoPixelFormat}} or when {{platform object/[[Detached]]}} is
    `true`.

    The {{VideoFrame/format}} getter steps are to return
    {{VideoFrame/[[format]]}}.

: <dfn attribute for=VideoFrame>codedWidth</dfn>
:: Width of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

    The {{VideoFrame/codedWidth}} getter steps are to return
    {{VideoFrame/[[coded width]]}}.

: <dfn attribute for=VideoFrame>codedHeight</dfn>
:: Height of the {{VideoFrame}} in pixels, potentially including non-visible
    padding, and prior to considering potential ratio adjustments.

    The {{VideoFrame/codedHeight}} getter steps are to return
    {{VideoFrame/[[coded height]]}}.

: <dfn attribute for=VideoFrame>codedRect</dfn>
:: A {{DOMRectReadOnly}} with {{DOMRectReadOnly/width}} and
    {{DOMRectReadOnly/height}} matching {{VideoFrame/codedWidth}} and
    {{VideoFrame/codedHeight}} and {{DOMRectReadOnly/x}} and
    {{DOMRectReadOnly/y}} at `(0,0)`. Offered for convenience for use with
    {{VideoFrame/allocationSize()}} and {{VideoFrame/copyTo()}}.

    The {{VideoFrame/codedRect}} getter steps are:
    1. If {{platform object/[[Detached]]}} is `true`, return `null`.
    2. Let |rect| be a new {{DOMRectReadOnly}}, initialized as follows:
        1. Assign `0` to {{DOMRectReadOnly/x}} and {{DOMRectReadOnly/y}}.
        2. Assign {{VideoFrame/[[coded width]]}} and
            {{VideoFrame/[[coded height]]}} to {{DOMRectReadOnly/width}} and
            {{DOMRectReadOnly/height}} respectively.
    3. Return |rect|.

: <dfn attribute for=VideoFrame>visibleRect</dfn>
:: A {{DOMRectReadOnly}} describing the visible rectangle of pixels for this
    {{VideoFrame}}.

    The {{VideoFrame/visibleRect}} getter steps are:
    1. If {{platform object/[[Detached]]}} is `true`, return `null`.
    2. Let |rect| be a new {{DOMRectReadOnly}}, initialized as follows:
        1. Assign {{VideoFrame/[[visible left]]}},
            {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
            and {{VideoFrame/[[visible height]]}} to {{DOMRectReadOnly/x}},
            {{DOMRectReadOnly/y}}, {{DOMRectReadOnly/width}}, and
            {{DOMRectReadOnly/height}} respectively.
    3. Return |rect|.

: <dfn attribute for=VideoFrame>displayWidth</dfn>
:: Width of the VideoFrame when displayed after applying aspect ratio
    adjustments.

    The {{VideoFrame/displayWidth}} getter steps are to return
    {{VideoFrame/[[display width]]}}.

: <dfn attribute for=VideoFrame>displayHeight</dfn>
:: Height of the VideoFrame when displayed after applying aspect ratio
    adjustments.

    The {{VideoFrame/displayHeight}} getter steps are to return
    {{VideoFrame/[[display height]]}}.

: <dfn attribute for=VideoFrame>timestamp</dfn>
:: The presentation timestamp, given in microseconds. The timestamp is copied
    from the {{EncodedVideoChunk}} corresponding to this VideoFrame.

    The {{VideoFrame/timestamp}} getter steps are to return
    {{VideoFrame/[[timestamp]]}}.

: <dfn attribute for=VideoFrame>duration</dfn>
:: The presentation duration, given in microseconds. The duration is copied
    from the {{EncodedVideoChunk}} corresponding to this VideoFrame.

    The {{VideoFrame/duration}} getter steps are to return
    {{VideoFrame/[[duration]]}}.

: <dfn attribute for=VideoFrame>colorSpace</dfn>
:: The {{VideoColorSpace}} associated with this frame.

    The {{VideoFrame/colorSpace}} getter steps are to return
    {{VideoFrame/[[color space]]}}.

### Internal Structures ###{#videoframe-internal-structures}
A <dfn>combined buffer layout</dfn> is a [=struct=] that consists of:
  * A <dfn for="combined buffer layout">allocationSize</dfn> (an {{unsigned
    long}})
  * A <dfn for="combined buffer layout">computedLayouts</dfn> (a [=list=] of
    [=computed plane layout=] structs).

A <dfn>computed plane layout</dfn> is a [=struct=] that consists of:
  * A <dfn for="computed plane layout">destinationOffset</dfn> (an
    {{unsigned long}})
  * A <dfn for="computed plane layout">destinationStride</dfn> (an
    {{unsigned long}})
  * A <dfn for="computed plane layout">sourceTop</dfn> (an {{unsigned long}})
  * A <dfn for="computed plane layout">sourceHeight</dfn> (an {{unsigned long}})
  * A <dfn for="computed plane layout">sourceLeftBytes</dfn> (an
    {{unsigned long}})
  * A <dfn for="computed plane layout">sourceWidthBytes</dfn> (an
    {{unsigned long}})

### Methods ###{#videoframe-methods}
: <dfn method for=VideoFrame>allocationSize(|options|)</dfn>
:: Returns the minimum byte length for a valid destination {{BufferSource}}
    to be used with {{VideoFrame/copyTo()}} with the given options.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. If {{VideoFrame/[[format]]}} is `null`, throw a {{NotSupportedError}}
        {{DOMException}}.
    3. Let |combinedLayout| be the result of running the [=Parse
        VideoFrameCopyToOptions=] algorithm with |options|.
    4. If |combinedLayout| is an exception, throw |combinedLayout|.
    5. Return |combinedLayout|'s [=combined buffer layout/allocationSize=].

: <dfn method for=VideoFrame>copyTo(|destination|, |options|)</dfn>
:: Asynchronously copies the planes of this frame into |destination| according
    to |options|. The format of the data is the same as this {{VideoFrame}}'s
    {{VideoFrame/format}}.

    NOTE: Promises that are returned by several calls to
        {{VideoFrame/copyTo()}} are not guaranteed to resolve in the order they
        were returned.

    When invoked, run these steps:
    1. If {{platform object/[[Detached]]}} is `true`, throw an
        {{InvalidStateError}} {{DOMException}}.
    2. If {{VideoFrame/[[format]]}} is `null`, throw a {{NotSupportedError}}
        {{DOMException}}.
    3. Let |combinedLayout| be the result of running the [=Parse
        VideoFrameCopyToOptions=] algorithm with |options|.
    4. If |combinedLayout| is an exception, return a promise rejected with
        |combinedLayout|.
    5. If `destination.byteLength` is less than |combinedLayout|'s [=combined
        buffer layout/allocationSize=], return a promise rejected with a
        {{TypeError}}.
    6. Let |p| be a new {{Promise}}.
    7. Let |copyStepsQueue| be the result of starting a new [=parallel queue=].
    8. Enqueue the following steps to |copyStepsQueue|:
        1. Let resource be the [=media resource=] referenced by
            [[resource reference]].
        2. Let |numPlanes| be the number of planes as defined by
            {{VideoFrame/[[format]]}}.
        3. Let |planeIndex| be `0`.
        4. While |planeIndex| is less than |combinedLayout|'s |numPlanes|:
            1. Let |sourceStride| be the stride of the plane in |resource| as
                identified by |planeIndex|.
            2. Let |computedLayout| be the [=computed plane layout=] in
                |combinedLayout|'s [=combined buffer layout/computedLayouts=] at
                the position of |planeIndex|
            3. Let |sourceOffset| be the product of multiplying
                |computedLayout|'s [=computed plane layout/sourceTop=] by
                |sourceStride|
            4. Add |computedLayout|'s [=computed plane layout/sourceLeftBytes=]
                to sourceOffset.
            5. Let |destinationOffset| be |computedLayout|'s
                [=computed plane layout/destinationOffset=].
            6. Let |rowBytes| be |computedLayout|'s
                [=computed plane layout/sourceWidthBytes=].
            7. Let |row| be `0`.
            8. While |row| is less than |computedLayout|'s
                [=computed plane layout/sourceHeight=]:
                1. Copy |rowBytes| bytes from |resource| starting at
                    |sourceOffset| to |destination| starting at
                    |destinationOffset|.
                2. Increment |sourceOffset| by |sourceStride|.
                3. Increment |destinationOffset| by |computedLayout|'s
                    [=computed plane layout/destinationStride=].
                4. Increment |row| by `1`.
            9. Increment |planeIndex| by `1`.
        5. Queue a task on the [=control thread=] event loop to resolve |p|.
    9. Return |p|.

: <dfn method for=VideoFrame>clone()</dfn>
:: Creates a new {{VideoFrame}} with a reference to the same
    [=media resource=].

    When invoked, run these steps:
    1. If the value of |frame|’s {{platform object/[[Detached]]}} internal slot is
        `true`, throw an {{InvalidStateError}} {{DOMException}}.
    2. Return the result of running the [=Clone VideoFrame=] algorithm with
        [=this=].

: <dfn method for=VideoFrame>close()</dfn>
:: Clears all state and releases the reference to the [=media resource=].
    Close is final.

    When invoked, run the [=Close VideoFrame=] algorithm with [=this=].

### Algorithms ###{#videoframe-algorithms}
: <dfn>Create a VideoFrame</dfn> (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, and |colorSpace|)
:: 1. Let |frame| be a new {{VideoFrame}}, constructed as follows:
        1. Assign `false` to {{platform object/[[Detached]]}}.
        2. Let |resource| be the [=media resource=] described by |output|.
        3. Let |resourceReference| be a reference to |resource|.
        4. Assign |resourceReference| to {{VideoFrame/[[resource reference]]}}.
        5. If |output| uses a recognized {{VideoPixelFormat}}, assign that format to
            {{VideoFrame/[[format]]}}. Otherwise, assign `null` to
            {{VideoFrame/[[format]]}}.
        6. Let |codedWidth| and |codedHeight| be the coded width and height of the
            |output| in pixels.
        8. Let |visibleLeft|, |visibleTop|, |visibleWidth|, and |visibleHeight| be
            the left, top, width and height for the visible rectangle of |output|.
        7. Let |displayWidth| and |displayHeight| be the the display size of
            |output| in pixels.
        8. If |displayAspectWidth| and |displayAspectHeight| are provided,
            increase |displayWidth| or |displayHeight| until the ratio of
            |displayWidth| to |displayHeight| matches the ratio of
            |displayAspectWidth| to |displayAspectHeight|.
        9. Assign |codedWidth|, |codedHeight|, |visibleLeft|, |visibleTop|,
            |visibleWidth|, |visibleHeight|, |displayWidth|, and
            |displayHeight| to {{VideoFrame/[[coded width]]}},
            {{VideoFrame/[[coded height]]}}, {{VideoFrame/[[visible left]]}},
            {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
            and {{VideoFrame/[[visible height]]}} respectively.
        10. Assign |duration| and |timestamp| to {{VideoFrame/[[duration]]}} and
            {{VideoFrame/[[timestamp]]}} respectively.
        11. Assign {{VideoFrame/[[color space]]}} with the result of running the
            [=VideoFrame/Pick Color Space=] algorithm, with |colorSpace| and
            {{VideoFrame/[[format]]}}.
    2. Return |frame|.


: <dfn for=VideoFrame>Pick Color Space</dfn> (with |overrideColorSpace| and |format|)
:: 1. If |overrideColorSpace| is provided, return a new {{VideoColorSpace}}
        constructed with |overrideColorSpace|.

        User Agents MAY replace `null` members of the provided
        |overrideColorSpace| with guessed values as determined by implementer
        defined heuristics.

    2. Otherwise, if {{VideoFrame/[[format]]}} is an [=RGB format=] return a new
        instance of the [=sRGB Color Space=]
    3. Otherwise, return a new instance of the [=REC709 Color Space=].

: <dfn>Validate VideoFrameInit</dfn> (with |format|, |codedWidth|, and
    |codedHeight|):
:: 1. If {{VideoFrameInit/visibleRect}} [=map/exists=]:
        1. Let |validAlignment| be the result of running the
            [=VideoFrame/Verify Rect Offset Alignment=] with |format| and
            |visibleRect|.
        2. If |validAlignment| is `false`, return `false`.
        3. If any attribute of {{VideoFrameInit/visibleRect}} is negative or
            not finite, return `false`.
        4. If {{VideoFrameInit/visibleRect}}.{{DOMRectInit/width}} == `0` or
            {{VideoFrameInit/visibleRect}}.{{DOMRectInit/height}} == `0` return
            `false`.
        5. If {{VideoFrameInit/visibleRect}}.{{DOMRectInit/y}} +
            {{VideoFrameInit/visibleRect}}.{{DOMRectInit/height}} >=
            |codedHeight|, return `false`.
        6. If {{VideoFrameInit/visibleRect}}.{{DOMRectInit/x}} +
            {{VideoFrameInit/visibleRect}}.{{DOMRectInit/width}} >=
            |codedWidth|, return `false`.
    2. If |codedWidth| = 0 or |codedHeight| = 0,return `false`.
    3. If only one of {{VideoFrameInit/displayWidth}} or
        {{VideoFrameInit/displayHeight}} [=map/exists=], return `false`.
    4. If {{VideoFrameInit/displayWidth}} == `0` or
        {{VideoFrameInit/displayHeight}} == `0`, return `false`.
    5. Return `true`.

: To check if a {{VideoFrameBufferInit}} is a
    <dfn>valid VideoFrameBufferInit</dfn>, run these steps:
:: 1. If {{VideoFrameBufferInit/codedWidth}} = 0 or
        {{VideoFrameBufferInit/codedHeight}} = 0,return `false`.
    2. If any attribute of {{VideoFrameBufferInit/visibleRect}} is negative or
        not finite, return `false`.
    3. If {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/y}} +
        {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/height}} >=
        {{VideoFrameBufferInit/codedHeight}}, return `false`.
    4. If {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/x}} +
        {{VideoFrameBufferInit/visibleRect}}.{{DOMRectInit/width}} >=
        {{VideoFrameBufferInit/codedWidth}}, return `false`.
    5. If only one of {{VideoFrameBufferInit/displayWidth}} or
        {{VideoFrameBufferInit/displayHeight}} [=map/exists=], return `false`.
    6. If {{VideoFrameBufferInit/displayWidth}} = 0 or
        {{VideoFrameBufferInit/displayHeight}} = 0, return `false`.
    7. Return `true`.

: <dfn for=VideoFrame>Initialize Frame From Other Frame</dfn> (with |init|,
    |frame|, and |otherFrame|)
:: 1. Let |format| be |otherFrame|.{{VideoFrame/format}}.
    2. If |init|.{{VideoFrameInit/alpha}} is {{AlphaOption/discard}},
        assign |otherFrame|.{{VideoFrame/format}}'s [=equivalent opaque format=]
        |format|.
    1. Let |validInit| be the result of running the [=Validate VideoFrameInit=]
        algorithm with |format| and |otherFrame|'s
        {{VideoFrame/[[coded width]]}} and {{VideoFrame/[[coded height]]}}.
    2. If |validInit| is `false`, throw a  {{TypeError}}.
    3. Let |resource| be the [=media resource=] referenced by |otherFrame|'s
        {{VideoFrame/[[resource reference]]}}.
    4. Assign a new reference for |resource| to |frame|'s
        {{VideoFrame/[[resource reference]]}}.
    5. Assign the following attributes from |otherFrame| to |frame|:
        {{VideoFrame/codedWidth}}, {{VideoFrame/codedHeight}},
        {{VideoFrame/colorSpace}}.
    6. Let |defaultVisibleRect| be the result of performing the getter steps
        for {{VideoFrame/visibleRect}} on |otherFrame|.
    7. Let |defaultDisplayWidth|, and |defaultDisplayHeight| be |otherFrame|'s
        {{VideoFrame/[[display width]]}}, and {{VideoFrame/[[display height]]}}
        respectively.
    8. Run the [=VideoFrame/Initialize Visible Rect and Display Size=]
        algorithm with |init|, |frame|, |defaultVisibleRect|,
        |defaultDisplayWidth|, and |defaultDisplayHeight|.
    9. If {{VideoFrameInit/duration}} [=map/exists=] in |init|, assign it to
        |frame|.{{VideoFrame/duration}}. Otherwise, assign
        |otherFrame|.{{VideoFrame/duration}} to
        |frame|.{{VideoFrame/duration}}.
    10. If {{VideoFrameInit/timestamp}} [=map/exists=] in |init|, assign it to
        |frame|.{{VideoFrame/timestamp}}. Otherwise, assign
        |otherFrame|.{{VideoFrame/timestamp}} to
        |frame|.{{VideoFrame/timestamp}}.
    11. Assign |format| to |frame|.{{VideoFrame/[[format]]}}.

: <dfn for=VideoFrame>Initialize Frame With Resource and Size</dfn> (with
    |init|,  |frame|, |resource|, |width| and |height|)
:: 1. Let |format| be `null`.
    2. If |resource| uses a recognized {{VideoPixelFormat}}, assign the
        {{VideoPixelFormat}} of |resource| to |format|.
    3. Let |validInit| be the result of running the [=Validate VideoFrameInit=]
        algorithm with |format|, |width| and |height|.
    4. If |validInit| is `false`, throw a  {{TypeError}}.
    5. Assign a new reference for |resource| to |frame|'s
        {{VideoFrame/[[resource reference]]}}.
    6. If |init|.{{VideoFrameInit/alpha}} is {{AlphaOption/discard}}, assign
        |format|'s [=equivalent opaque format=] to |format|.
    7. Assign |format| to {{VideoFrame/[[format]]}}
    8. Assign |width| and |height| to |frame|'s {{VideoFrame/[[coded width]]}}
        and {{VideoFrame/[[coded height]]}} respectively.
    9. Let |defaultVisibleRect| be a new {{DOMRect}} constructed with
        «[ "x:" → `0`, "y" → `0`, "width" → |width|, "height" → |height| ]»
    10. Run the [=VideoFrame/Initialize Visible Rect and Display Size=]
        algorithm with |init|, |frame|, |defaultVisibleRect|, |width|, and
        |height|.
    11. Assign `init`.{{VideoFrameInit/duration}} to
        |frame|.{{VideoFrame/duration}}.
    12. Assign `init`.{{VideoFrameInit/timestamp}} to
        |frame|.{{VideoFrame/timestamp}}.
    13. If |resource| has a known {{VideoColorSpace}}, assign its value to
        {{VideoFrame/[[color space]]}}.
    14. Otherwise, assign a new {{VideoColorSpace}}, constructed with an empty
        {{VideoColorSpaceInit}}, to {{VideoFrame/[[color space]]}}.

: <dfn for=VideoFrame>Initialize Visible Rect and Display Size</dfn> (with
    |init|, |frame|, |defaultVisibleRect|, |defaultDisplayWidth| and
    |defaultDisplayHeight|)
:: 1. Let |visibleRect| be |defaultVisibleRect|.
    2. If |init|.{{VideoFrameInit/visibleRect}} [=map/exists=], assign it to
        |visibleRect|.
    3. Assign |visibleRect|'s {{DOMRect/x}}, {{DOMRect/y}}, {{DOMRect/width}},
        and {{DOMRect/height}}, to |frame|'s {{VideoFrame/[[visible left]]}},
        {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}}, and
        {{VideoFrame/[[visible height]]}} respectively.
    4. If {{VideoFrameInit/displayWidth}} and {{VideoFrameInit/displayHeight}}
        [=map/exist=] in |init|, assign them to {{VideoFrame/[[display width]]}}
        and {{VideoFrame/[[display height]]}} respectively.
    5. Otherwise:
        1. Let |widthScale| be the result of dividing |defaultDisplayWidth| by
            |defaultVisibleRect|.{{DOMRect/width}}.
        2. Let |heightScale| be the result of dividing |defaultDisplayHeight| by
            |defaultVisibleRect|.{{DOMRect/height}}.
        3. Multiply |frame|'s {{VideoFrame/[[visible width]]}} by |widthScale|
            and round the result. Assign the rounded result to
            {{VideoFrame/[[display width]]}}.
        4. Multiply |frame|'s {{VideoFrame/[[visible height]]}} by
            |heightScale| and round the result. Assign the rounded result to
            |frame|'s {{VideoFrame/[[display height]]}}.

: <dfn>Clone VideoFrame</dfn> (with |frame|)
:: 1. Let |clone| be a new {{VideoFrame}} initialized as follows:
        1. Let |resource| be the [=media resource=] referenced by |frame|’s
            [[resource reference]].
        2. Let |newReference| be a new reference to |resource|.
        3. Assign |newReference| to |clone|'s
            {{VideoFrame/[[resource reference]]}}.
        4. Assign all remaining internal slots of |frame| (excluding
            {{VideoFrame/[[resource reference]]}}) to those of the same name
            in |clone|.
    2. Return |clone|.

: <dfn>Close VideoFrame</dfn> (with |frame|)
:: 1. Assign `null` to |frame|'s {{VideoFrame/[[resource reference]]}}.
    2. Assign `true` to |frame|'s {{platform object/[[Detached]]}}.
    3. Assign `null` to |frame|'s {{VideoFrame/format}}.
    4. Assign `0` to |frame|'s {{VideoFrame/[[coded width]]}},
        {{VideoFrame/[[coded height]]}}, {{VideoFrame/[[visible left]]}},
        {{VideoFrame/[[visible top]]}}, {{VideoFrame/[[visible width]]}},
        {{VideoFrame/[[visible height]]}}, {{VideoFrame/[[display width]]}},
        and {{VideoFrame/[[display height]]}}.
    5. Assign `null` to |frame|'s {{VideoFrame/[[duration]]}} and
        {{VideoFrame/[[timestamp]]}}.

: <dfn for=VideoFrame>Parse VideoFrameCopyToOptions</dfn> (with |options|)
::  1. Let |defaultRect| be the result of performing the getter steps for
        {{VideoFrame/visibleRect}}.
    2. Let |overrideRect| be `undefined`.
    3. If |options|.{{VideoFrameCopyToOptions/rect}} [=map/exists=]:
        1. Assign the value of |options|.{{VideoFrameCopyToOptions/rect}} to
            |overrideRect|.
        2. Let |validAlignment| be the result of running the
            [=VideoFrame/Verify Rect Size Alignment=] algorithm with
            |overrideRect| and {{VideoFrame/[[format]]}}.
        3. If |validAlignment| is `false`, throw a {{TypeError}}.
    4. Let |parsedRect| be the result of running the [=VideoFrame/Parse Visible
        Rect=] algorithm with |defaultRect|, |overrideRect|,
        {{VideoFrame/[[coded width]]}}, {{VideoFrame/[[coded height]]}}, and
        {{VideoFrame/[[format]]}}.
    5. If |parsedRect| is an exception, return |parsedRect|.
    6. Let |optLayout| be `undefined`.
    7. If |options|.{{VideoFrameCopyToOptions/layout}} [=map/exists=], assign
        its value to |optLayout|.
    8. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
        Layout and Allocation Size=] algorithm with |parsedRect|,
        {{VideoFrame/[[format]]}}, and |optLayout|.
    9. Return |combinedLayout|.

: <dfn for=VideoFrame>Verify Rect Offset Alignment</dfn> (with |format| and
    |rect|)
:: 1. If |format| is `null`, return `true`.
    2. Let |planeIndex| be `0`.
    3. Let |numPlanes| be the number of planes as defined by |format|.
    4. While |planeIndex| is less than |numPlanes|:
        1. Let |plane| be the Plane identified by |planeIndex| as defined by
            |format|.
        2. Let |sampleWidth| be the horizontal [=sub-sampling factor=] of each
            subsample for |plane|.
        3. Let |sampleHeight| be the vertical [=sub-sampling factor=] of each
            subsample for |plane|.
        4. If |rect|.{{DOMRectReadOnly/x}} is not a multiple of |sampleWidth|,
            return `false`.
        5. If |rect|.{{DOMRectReadOnly/y}} is not a multiple of |sampleHeight|, 
            return `false`.
        8. Increment |planeIndex| by `1`.
    5. Return `true`.

: <dfn for=VideoFrame>Verify Rect Size Alignment</dfn> (with |format| and
    |rect|)
:: 1. If |format| is `null`, return `true`.
    2. Let |planeIndex| be `0`.
    3. Let |numPlanes| be the number of planes as defined by |format|.
    4. While |planeIndex| is less than |numPlanes|:
        1. Let |plane| be the Plane identified by |planeIndex| as defined by
            |format|.
        2. Let |sampleWidth| be the horizontal [=sub-sampling factor=] of each
            subsample for |plane|.
        3. Let |sampleHeight| be the vertical [=sub-sampling factor=] of each
            subsample for |plane|.
        6. If |rect|.{{DOMRectReadOnly/width}} is not a multiple of
            |sampleWidth|, return `false`.
        7. If |rect|.{{DOMRectReadOnly/height}} is not a multiple of
            |sampleHeight|, return `false`.
        8. Increment |planeIndex| by `1`.
    5. Return `true`.

: <dfn for=VideoFrame>Parse Visible Rect</dfn> (with |defaultRect|,
    |overrideRect|, |codedWidth|, |codedHeight|, and |format|)
:: 1. Let |sourceRect| be |defaultRect|
    2. If |overrideRect| is not `undefined`:
        1. If either of |overrideRect|.{{DOMRectInit/width}} or
            {{DOMRectInit/height}} is `0`, return a {{TypeError}}.
        2. If the sum of |overrideRect|.{{DOMRectInit/x}} and
            |overrideRect|.{{DOMRectInit/width}} is greater than
            |codedWidth|, return a {{TypeError}}.
        3. If the sum of |overrideRect|.{{DOMRectInit/y}} and
            |overrideRect|.{{DOMRectInit/height}} is greater than
            |codedHeight|, return a {{TypeError}}.
        4. Assign |overrideRect| to |sourceRect|.
    3. Let |validAlignment| be the result of running the
        [=VideoFrame/Verify Rect Offset Alignment=] algorithm with |format| and
        |sourceRect|.
    4. If |validAlignment| is `false`, throw a {{TypeError}}.
    5. Return |sourceRect|.

: <dfn for=VideoFrame>Compute Layout and Allocation Size</dfn> (with
    |parsedRect|, |format|, and |layout|)
:: 1. Let |numPlanes| be the number of planes as defined by |format|.
    2. If |layout| is not `undefined` and its length does not equal |numPlanes|,
         throw a {{TypeError}}.
    3. Let |minAllocationSize| be `0`.
    4. Let |computedLayouts| be a new [=list=].
    5. Let |endOffsets| be a new [=list=].
    6. Let |planeIndex| be `0`.
    7. While |planeIndex| < |numPlanes|:
        1. Let |plane| be the Plane identified by |planeIndex| as defined by
            |format|.
        2. Let |sampleBytes| be the number of bytes per sample for |plane|.
        3. Let |sampleWidth| be the horizontal [=sub-sampling factor=] of each
            subsample for |plane|.
        4. Let |sampleHeight| be the vertical [=sub-sampling factor=] of each
            subsample for |plane|.
        5. Let |sampleWidthBytes| be the product of multiplying |sampleWidth| by
            |sampleBytes|.
        6. Let |computedLayout| be a new [=computed plane layout=].
        7. Set |computedLayout|'s [=computed plane layout/sourceTop=] to the
            result of the integer division of
            truncated |parsedRect|.{{DOMRectInit/y}} by |sampleHeight|.
        8. Set |computedLayout|'s [=computed plane layout/sourceHeight=] to the
            result of the integer division of
            truncated |parsedRect|.{{DOMRectInit/height}} by |sampleHeight|
        9. Set |computedLayout|'s [=computed plane layout/sourceLeftBytes=] to
            the result of the integer division of
            truncated |parsedRect|.{{DOMRectInit/x}} by |sampleWidthBytes|.
        10. Set |computedLayout|'s [=computed plane layout/sourceWidthBytes=] to
            the result of the integer division of
            truncated |parsedRect|.{{DOMRectInit/width}} by |sampleWidthBytes|.
        11. If |layout| is not `undefined`:
            1. Let |planeLayout| be the {{PlaneLayout}} in |layout| at position
                |planeIndex|.
            2. If |planeLayout|.{{PlaneLayout/stride}} is less than
                |computedLayout|'s [=computed plane layout/sourceWidthBytes=],
                return a {{TypeError}}.
            3. Assign |planeLayout|.{{PlaneLayout/offset}} to |computedLayout|'s
                [=computed plane layout/destinationOffset=].
            4. Assign |planeLayout|.{{PlaneLayout/stride}} to |computedLayout|'s
                [=computed plane layout/destinationStride=].
        12. Otherwise:

            NOTE: If an explicit layout was not provided, the following steps
                default to tight packing.

            1. Assign |minAllocationSize| to |computedLayout|'s
                [=computed plane layout/destinationOffset=].
            2. Assign |computedLayout|'s [=computed plane layout/sourceWidthBytes=] to
                |computedLayout|'s [=computed plane layout/destinationStride=].
        13. Let |planeSize| be the product of multiplying |computedLayout|'s
                [=computed plane layout/destinationStride=] and
                [=computed plane layout/sourceHeight=].
        14. Let |planeEnd| be the sum of |planeSize| and |computedLayout|'s
                [=computed plane layout/destinationOffset=].
        15. If |planeSize| or |planeEnd| is greater than maximum range of
                {{unsigned long}}, return a {{TypeError}}.
        16. Append |planeEnd| to |endOffsets|.
        17. Assign the maximum of |minAllocationSize| and |planeEnd| to
                |minAllocationSize|.

                NOTE: The above step uses a maximum to allow for the
                    possibility that user specified plane offsets reorder
                    planes.

        18. Let |earlierPlaneIndex| be `0`.
        19. While |earlierPlaneIndex| is less than |planeIndex|.
            1. Let |earlierLayout| be `computedLayouts[earlierPlaneIndex]`.
            2. If `endOffsets[planeIndex]` is less than or equal to
                |earlierLayout|'s [=computed plane layout/destinationOffset=] or
                if `endOffsets[earlierPlaneIndex]` is less than or equal to
                |computedLayout|'s [=computed plane layout/destinationOffset=],
                continue.

                NOTE: If plane A ends before plane B starts, they do not
                    overlap.

            3. Otherwise, return a {{TypeError}}.
            4. Increment |earlierPlaneIndex| by `1`.
        20. Append |computedLayout| to |computedLayouts|.
        21. Increment |planeIndex| by `1`.
    8. Let |combinedLayout| be a new [=combined buffer layout=], initialized as
        follows:
        1. Assign |computedLayouts| to
            [=combined buffer layout/computedLayouts=].
        2. Assign |minAllocationSize| to
            [=combined buffer layout/allocationSize=].
    9. Return |combinedLayout|.

### Transfer and Serialization ###{#videoframe-transfer-serialization}

: The {{VideoFrame}} [=transfer steps=] (with |value| and |dataHolder|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. For all {{VideoFrame}} internal slots in |value|, assign the value of
        each internal slot to a field in |dataHolder| with the same name as the
        internal slot.
    3. Run the [=Close VideoFrame=] algorithm with |value|.

: The {{VideoFrame}} [=transfer-receiving steps=] (with |dataHolder| and
    |value|) are:
:: 1. For all named fields in |dataHolder|, assign the value of each named
        field to the {{VideoFrame}} internal slot in |value| with the same name
        as the named field.

: The {{VideoFrame}} [=serialization steps=] (with |value|, |serialized|, and
    |forStorage|) are:
:: 1. If |value|'s {{platform object/[[Detached]]}} is `true`, throw a
        {{DataCloneError}} {{DOMException}}.
    2. If |forStorage| is `true`, throw a {{TypeError}}.
    3. Let |resource| be the [=media resource=] referenced by
            |value|'s {{VideoFrame/[[resource reference]]}}.
    4. Let |newReference| be a new reference to |resource|.
    5. Assign |newReference| to |serialized|.[[resource reference]].
    6. For all remaining {{VideoFrame}} internal slots (excluding
        {{VideoFrame/[[resource reference]]}}) in |value|, assign the value of
        each internal slot to a field in |serialized| with the same name as the
        internal slot.

: The {{VideoFrame}} [=deserialization steps=] (with |serialized| and |value|)
    are:
:: 1. For all named fields in |serialized|, assign the value of each named
        field to the {{VideoFrame}} internal slot in |value| with the same name
        as the named field.

### Rendering ### {#videoframe-rendering}

When rendered, for example by {{CanvasDrawImage}}
{{CanvasDrawImage/drawImage()}}, a {{VideoFrame}} <em class="rfc2119">MUST</em>
be converted to a color space compatible with the rendering target, unless color
conversion is explicitly disabled.

Color space conversion during {{ImageBitmap}} construction is controlled by
{{ImageBitmapOptions}} {{ImageBitmapOptions/colorSpaceConversion}}. Setting this
value to "none" disables color space conversion.

VideoFrame CopyTo() Options {#videoframe-copyto-options}
------------------------------------------------------------
Options to specify a rectangle of pixels to copy and the offset and stride of
planes in the destination buffer.

<xmp class='idl'>
dictionary VideoFrameCopyToOptions {
  DOMRectInit rect;
  sequence<PlaneLayout> layout;
};
</xmp>

<div class='note'>
NOTE: The steps of {{VideoFrame/copyTo()}} or {{VideoFrame/allocationSize()}}
will enforce the following requirements:
    * The coordinates of {{VideoFrameCopyToOptions/rect}} are
        sample-aligned as determiend by {{VideoFrame/[[format]]}}.
    * If {{VideoFrameCopyToOptions/layout}} [=map/exists=], a {{PlaneLayout}}
        is provided for all planes.
</div>

: <dfn dict-member for=VideoFrameCopyToOptions>rect</dfn>
:: A {{DOMRectInit}} describing the rectangle of pixels to copy from the
    {{VideoFrame}}. If unspecified, the {{VideoFrame/visibleRect}} will be used.

    NOTE: The coded rectangle can be specified by passing {{VideoFrame}}'s
        {{VideoFrame/codedRect}}.

    NOTE: The default {{VideoFrameCopyToOptions/rect}} does not necessarily meet
        the sample-alignment requirement and can result in
        {{VideoFrame/copyTo()}} or {{VideoFrame/allocationSize()}} rejecting.
: <dfn dict-member for=VideoFrameCopyToOptions>layout</dfn>
:: The {{PlaneLayout}} for each plane in {{VideoFrame}}, affording the option
    to specify an offset and stride for each plane in the destination
    {{BufferSource}}. If unspecified, the planes will be tightly packed. It is
    invalid to specify planes that overlap.

DOMRects in VideoFrame {#videoframe-domrect}
--------------------------------------------
The {{VideoFrame}} interfaces uses {{DOMRect}}s to specify the position and
dimensions for a rectangle of pixels. {{DOMRectInit}} is used with
{{VideoFrame/copyTo()}} and {{VideoFrame/allocationSize()}} to describe the
dimensions of the source rectangle. {{VideoFrame}} defines
{{VideoFrame/codedRect}} and {{VideoFrame/visibleRect}} for convenient copying
of the coded size and visible region respectively.

NOTE: VideoFrame pixels are only addressable by integer numbers. All floating
    point values provided to {{DOMRectInit}} will be truncated.

Plane Layout{#plane-layout}
---------------------------
A {{PlaneLayout}} is a dictionary specifying the offset and stride of a
{{VideoFrame}} plane once copied to a {{BufferSource}}. A sequence of
{{PlaneLayout}}s <em class="rfc2119">MAY</em> be provided to {{VideoFrame}}'s
{{VideoFrame/copyTo()}} to specify how the plane is laid out in the destination
{{BufferSource}}}.  Alternatively, callers can inspect {{VideoFrame/copyTo()}}'s
returned sequence of {{PlaneLayout}}s to learn the the offset and stride for
planes as decided by the User Agent.

<xmp class='idl'>
dictionary PlaneLayout {
  required [EnforceRange] unsigned long offset;
  required [EnforceRange] unsigned long stride;
};
</xmp>

: <dfn dict-member for=PlaneLayout>offset</dfn>
:: The offset in bytes where the given plane begins within a {{BufferSource}}.

: <dfn dict-member for=PlaneLayout>stride</dfn>
:: The number of bytes, including padding, used by each row of the plane within
    a {{BufferSource}}.

Pixel Format{#pixel-format}
---------------------------
Pixel formats describe the arrangement of bytes in each plane as well as the
number and order of the planes. Each format is described in its own sub-section.

<xmp class='idl'>
enum VideoPixelFormat {
  // 4:2:0 Y, U, V
  "I420",
  // 4:2:0 Y, U, V, A
  "I420A",
  // 4:2:2 Y, U, V
  "I422",
  // 4:4:4 Y, U, V
  "I444",
  // 4:2:0 Y, UV
  "NV12",
  // 32bpp RGBA
  "RGBA",
  // 32bpp RGBX (opaque)
  "RGBX",
  // 32bpp BGRA
  "BGRA",
  // 32bpp BGRX (opaque)
  "BGRX",
};
</xmp>

<dfn lt="sub-sampling|sub-sampled">Sub-sampling</dfn> is a technique
where a single sample contains information for multiple pixels in the final
image. [=Sub-sampling=] can be horizontal, vertical or both, and has a <dfn
lt="sub-sampling factor|factor">factor</dfn>, that is the number of final pixels
in the image that are derived from a [=sub-sampled=] sample.

<div class=example>
  If a {{VideoFrame}} is in {{I420}} format, then the very first
  component of the second plane (the U plane) corresponds to four pixels, that are
  the pixels in the top-left angle of the image. Consequently, the first
  component of the second row corresponds to the four pixels below those initial
  four top-left pixels. The [=sub-sampling factor=] is 2 in both the horizontal
  and vertical direction.
</div>

If a {{VideoPixelFormat}} has an alpha component, the format's
<dfn>equivalent opaque format</dfn> is the same {{VideoPixelFormat}}, without an
alpha component. If a {{VideoPixelFormat}} does not have an alpha component, it
is its own [=equivalent opaque format=].

<dl>
  <dt><dfn enum-value for=VideoPixelFormat>I420</dfn></dt>
  <dd>
    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order. It is also
    often refered to as Planar YUV 4:2:0.

    The U an V planes are [=sub-sampled=] horizontaly and vertically by a
    [=factor=] of 2 compared to the Y plane.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples
    (and therefore bytes) in the Y plane, arranged starting at the top left in
    the image, in {{VideoFrame/codedHeight}} lines of {{VideoFrame/codedWidth}}
    samples.

    There is {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} / 4 samples
    (and therefore bytes) in the two U and V planes, arranged starting at the
    top left in the image, in {{VideoFrame/codedHeight}} / 2 lines of
    {{VideoFrame/codedWidth}} / 2 samples.

    The {{VideoFrame/codedWidth}} and {{VideoFrame/codedHeight}} MUST be even.
    Similarly, the visible rectangle offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}} and
    {{VideoFrame/visibleRect}}.{{DOMRectInit/y}}) MUST be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I420A</dfn></dt>
  <dd>

    This format is composed of four distinct planes, one plane of Luma, two
    planes of Chroma, denoted Y, U and V, and one place of alpha values, all
    present in this order. It is also often refered to as Planar YUV 4:2:0 with
    an alpha channel.

    The U an V planes are [=sub-sampled=] horizontaly and vertically by a
    [=factor=] of 2 compared to the Y and Alpha planes.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples (and
    therefore bytes) in the Y and alpha plane, arranged starting at the top left
    in the image, in {{VideoFrame/codedHeight}} lines of
    {{VideoFrame/codedWidth}} samples.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} / 4 samples
    (and therefore bytes) in the two U and V planes, arranged starting at the
    top left in the image, in {{VideoFrame/codedHeight}} / 2 lines of
    {{VideoFrame/codedWidth}} / 2 samples.

    The {{VideoFrame/codedWidth}} and {{VideoFrame/codedHeight}} MUST be even.
    Similarly, the visible rectangle offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}} and
    {{VideoFrame/visibleRect}}.{{DOMRectInit/y}}) MUST be even.

    {{I420A}}'s [=equivalent opaque format=] is {{I420}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I422</dfn></dt>
  <dd>

    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order. It is also
    often refered to as Planar YUV 4:2:2.

    The U an V planes are [=sub-sampled=] horizontaly by a [=factor=] of 2
    compared to the Y plane, and not [=sub-sampled=] vertically.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples (and
    therefore bytes) in the Y plane, arranged starting at the top left in the
    image, in {{VideoFrame/codedHeight}} lines of {{VideoFrame/codedWidth}}
    samples.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} / 2 samples
    (and therefore bytes) in the two U and V planes, arranged starting at the
    top left in the image, in {{VideoFrame/codedHeight}} / 2 lines of
    {{VideoFrame/codedWidth}} samples.

    The {{VideoFrame/codedHeight}} MUST be even. Similarly, the visible
    rectangle offset ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}} and
    {{VideoFrame/visibleRect}}.{{DOMRectInit/y}}) MUST be even.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>I444</dfn></dt>
  <dd>

    This format is composed of three distinct planes, one plane of Luma and two
    planes of Chroma, denoted Y, U and V, and present in this order. It is also
    often refered to as Planar YUV 4:4:4.

    Each sample in this format is 8 bits. This format does not use
    [=sub-sampling=].

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples (and
    therefore bytes) in all three planes, arranged starting at the top left in the
    image, in {{VideoFrame/codedHeight}} lines of {{VideoFrame/codedWidth}}
    samples.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>NV12</dfn></dt>
  <dd>

    This format is composed of two distinct planes, one plane of Luma and then
    another plane for the two Chroma components. The two planes are present in
    this order, and are refered to as respectively the Y plane and the UV plane.

    The U an V components are [=sub-sampled=] horizontaly and vertically by a
    [=factor=] of 2 compared to the components in the Y planes.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} samples (and
    therefore bytes) in the Y plane, arranged starting at the top left in the
    image, in {{VideoFrame/codedHeight}} lines of {{VideoFrame/codedWidth}}
    samples.

    The UV planes is composed of interleaved U and V values, in
    {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} / 4 elements (of two
    bytes each), arranged starting at the top left in the image, in
    {{VideoFrame/codedHeight}} / 2 lines of {{VideoFrame/codedWidth}} / 2
    elements. Each element is composed of a two Chroma values, the U and V
    value, in this order.

    The {{VideoFrame/codedWidth}} and {{VideoFrame/codedHeight}} MUST be even.
    Similarly, the visible rectangle offset
    ({{VideoFrame/visibleRect}}.{{DOMRectInit/x}} and
    {{VideoFrame/visibleRect}}.{{DOMRectInit/y}}) MUST be even.

    <div class=example>
    An image in the NV12 pixel format that is 16 pixels wide and 9 pixels tall
    will be arranged like so in memory:

    ```
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        YYYYYYYYYYYYYY
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
        UVUVUVUVUVUVUV
    ```

    All samples being linear in memory.
  </div>
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>RGBA</dfn></dt>
  <dd>

    This format is composed of a single plane, that encodes four components:
    Red, Green, Blue, and an alpha value, present in this order.

    Each sample in this format is 8 bits, and each pixel is therefore 32 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top
    left in the image, in {{VideoFrame/codedHeight}} lines of
    {{VideoFrame/codedWidth}} samples.

    {{RGBA}}'s [=equivalent opaque format=] is {{RGBX}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>RGBX</dfn></dt>
  <dd>

    This format is composed of a single plane, that encodes four components:
    Red, Green, Blue, and a padding value, present in this order.

    Each sample in this format is 8 bits. The fourth element in each pixel is to
    be ignored, the image is always fully opaque.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top left
    in the image, in {{VideoFrame/codedHeight}} lines of
    {{VideoFrame/codedWidth}} samples.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>BGRA</dfn></dt>
  <dd>

    This format is composed of a single plane, that encodes four components:
    Blue, Green, Red, and an alpha value, present in this order.

    Each sample in this format is 8 bits.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top left
    in the image, in {{VideoFrame/codedHeight}} lines of
    {{VideoFrame/codedWidth}} samples.

    {{BGRA}}'s [=equivalent opaque format=] is {{BGRX}}.
  </dd>
  <dt><dfn enum-value for=VideoPixelFormat>BGRX</dfn></dt>
  <dd>

    This format is composed of a single plane, that encodes four components:
    Blue, Green, Red, and a padding value, present in this order.

    Each sample in this format is 8 bits. The fourth element in each pixel is to
    be ignored, the image is always fully opaque.

    There are {{VideoFrame/codedWidth}} * {{VideoFrame/codedHeight}} * 4 samples
    (and therefore bytes) in the single plane, arranged starting at the top left
    in the image, in {{VideoFrame/codedHeight}} lines of
    {{VideoFrame/codedWidth}} samples.
  </dd>
</dl>

Video Color Space Interface {#video-color-space-interface}
----------------------------------------------------------

<xmp class='idl'>
[Exposed=(Window,DedicatedWorker)]
interface VideoColorSpace {
  constructor(optional VideoColorSpaceInit init = {});

  readonly attribute VideoColorPrimaries? primaries;
  readonly attribute VideoTransferCharacteristics? transfer;
  readonly attribute VideoMatrixCoefficients? matrix;
  readonly attribute boolean? fullRange;

  [Default] VideoColorSpaceInit toJSON();
};

dictionary VideoColorSpaceInit {
  VideoColorPrimaries primaries;
  VideoTransferCharacteristics transfer;
  VideoMatrixCoefficients matrix;
  boolean fullRange;
};
</xmp>

### Internal Slots ### {#videocolorspace-internal-slots}
: <dfn attribute for=VideoColorSpace>\[[primaries]]</dfn>
:: The color primaries.
: <dfn attribute for=VideoColorSpace>\[[transfer]]</dfn>
:: The transfer characteristics.
: <dfn attribute for=VideoColorSpace>\[[matrix]]</dfn>
:: The matrix coefficients.
: <dfn attribute for=VideoColorSpace>[[full range]]</dfn>
:: Indicates whether full-range color values are used.

### Constructors ### {#videocolorspace-constructors}
<dfn constructor for=VideoColorSpace title="VideoColorSpace(init)">
  VideoColorSpace(init)
</dfn>
1. Let |c| be a new {{VideoColorSpace}} object, initialized as follows:
    1. If `primaries` is present in `init`, assign `init.primaries` to
        {{VideoColorSpace/[[primaries]]}}. Otherwise, assign `null` to
        {{VideoColorSpace/[[primaries]]}}.
    2. If `transfer` is present in `init`, assign `init.transfer` to
        {{VideoColorSpace/[[transfer]]}}. Otherwise, assign `null` to
        {{VideoColorSpace/[[transfer]]}}.
    3. If `matrix` is present in `init`, assign `init.matrix` to
        {{VideoColorSpace/[[matrix]]}}. Otherwise, assign `null` to
        {{VideoColorSpace/[[matrix]]}}.
    4. If `fullRange` is present in `init`, assign `init.fullRange` to
        {{VideoColorSpace/[[full range]]}}. Otherwise, assign `null` to
        {{VideoColorSpace/[[full range]]}}.
6. Return |c|.

### Attributes ### {#videocolorspace-attributes}
: <dfn attribute for=VideoColorSpace>primaries</dfn>
:: The {{VideoColorSpace/primaries}} getter steps are to return the value of
    {{VideoColorSpace/[[primaries]]}}.
: <dfn attribute for=VideoColorSpace>transfer</dfn>
:: The {{VideoColorSpace/transfer}} getter steps are to return the value of
    {{VideoColorSpace/[[transfer]]}}.
: <dfn attribute for=VideoColorSpace>matrix</dfn>
:: The {{VideoColorSpace/matrix}} getter steps are to return the value of
    {{VideoColorSpace/[[matrix]]}}.
: <dfn attribute for=VideoColorSpace>fullRange</dfn>
:: The {{VideoColorSpace/fullRange}} getter steps are to return the value of
    {{VideoColorSpace/[[full range]]}}.

Video Color Primaries {#videocolorprimaries}
--------------------------------------------
Color primaries describe the color gamut of video samples.

<xmp class='idl'>
enum VideoColorPrimaries {
  "bt709",      // BT.709, sRGB
  "bt470bg",    // BT.601 PAL
  "smpte170m",  // BT.601 NTSC
};
</xmp>

<dl>
  <dt><dfn enum-value for=VideoColorPrimaries>bt709</dfn></dt>
  <dd>
    Color primaries used by BT.709 and sRGB, as described by [[H.273]]
    section 8.1 table 2 value 1.
  </dd>
  <dt><dfn enum-value for=VideoColorPrimaries>bt470bg</dfn></dt>
  <dd>
    Color primaries used by BT.601 PAL, as described by [[H.273]]
    section 8.1 table 2 value 5.
  </dd>
  <dt><dfn enum-value for=VideoColorPrimaries>smpte170m</dfn></dt>
  <dd>
    Color primaries used by BT.601 NTSC, as described by [[H.273]]
    section 8.1 table 2 value 6.
  </dd>
</dl>

Video Transfer Characteristics {#videotransfercharacteristics}
--------------------------------------------------------------
Transfer characteristics describe the opto-electronic transfer characteristics
of video samples.

<xmp class='idl'>
enum VideoTransferCharacteristics {
  "bt709",         // BT.709
  "smpte170m",     // BT.601 (functionally the same as bt709)
  "iec61966-2-1",  // sRGB
};
</xmp>

<dl>
  <dt><dfn enum-value for=VideoTransferCharacteristics>bt709</dfn></dt>
  <dd>
    Transfer characteristics used by BT.709, as described by [[H.273]]
    section 8.2 table 3 value 1.
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>smpte170m</dfn></dt>
  <dd>
    Transfer characteristics used by BT.601, as described by [[H.273]]
    section 8.2 table 3 value 6.
  </dd>
  <dt><dfn enum-value for=VideoTransferCharacteristics>iec61966-2-1</dfn></dt>
  <dd>
    Transfer characteristics used by sRGB, as described by [[H.273]]
    section 8.2 table 3 value 13.
  </dd>
</dl>

Video Matrix Coefficients {#videomatrixcoefficients}
----------------------------------------------------
Matrix coefficients describe the relationship between sample component values
and color coordinates.

<xmp class='idl'>
enum VideoMatrixCoefficients {
  "rgb",        // sRGB
  "bt709",      // BT.709
  "bt470bg",    // BT.601 PAL
  "smpte170m",  // BT.601 NTSC (functionally the same as bt470bg)
};
</xmp>

<dl>
  <dt><dfn enum-value for=VideoMatrixCoefficients>rgb</dfn></dt>
  <dd>
    Matrix coefficients used by sRGB, as described by [[H.273]]
    section 8.3 table 4 value 0.
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>bt709</dfn></dt>
  <dd>
    Matrix coefficients used by BT.709, as described by [[H.273]]
    section 8.3 table 4 value 1.
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>bt470bg</dfn></dt>
  <dd>
    Matrix coefficients used by BT.601 PAL, as described by [[H.273]]
    section 8.3 table 4 value 5.
  </dd>
  <dt><dfn enum-value for=VideoMatrixCoefficients>smpte170m</dfn></dt>
  <dd>
    Matrix coefficients used by BT.601 NTSC, as described by [[H.273]]
    section 8.3 table 4 value 6.
  </dd>
</dl>


Image Decoding {#image-decoding}
====================================

Background {#image-decoding-background}
-------------------------------------

<div class=non-normative>
  This section is non-normative.

  Image codec definitions are typically accompanied by a definition for a
  corresponding file format. Hence image decoders often perform both duties of
  unpacking (demuxing) as well as decoding the encoded image data. The WebCodecs
  {{ImageDecoder}} follows this pattern, which motivates an interface design that
  is notably different from that of {{VideoDecoder}} and {{AudioDecoder}}.

  In spite of these differences, {{ImageDecoder}} uses the same
  [=codec processing model=] as the other codec interfaces. Additionally,
  {{ImageDecoder}} uses the {{VideoFrame}} interface to describe decoded outputs.
</div>

ImageDecoder Interface {#imagedecoder-interface}
------------------------------------------------

<pre class='idl'>
<xmp>
[Exposed=(Window,DedicatedWorker), SecureContext]
interface ImageDecoder {
  constructor(ImageDecoderInit init);

  readonly attribute DOMString type;
  readonly attribute boolean complete;
  readonly attribute Promise<undefined> completed;
  readonly attribute ImageTrackList tracks;

  Promise<ImageDecodeResult> decode(optional ImageDecodeOptions options = {});
  undefined reset();
  undefined close();

  static Promise<boolean> isTypeSupported(DOMString type);
};
</xmp>
</pre>

### Internal Slots ### {#imagedecoder-internal-slots}

: <dfn attribute for=ImageDecoder>\[[ImageTrackList]]</dfn>
:: An {{ImageTrackList}} describing the tracks found in
    {{ImageDecoder/[[encoded data]]}}

: <dfn attribute for=ImageDecoder>\[[type]]</dfn>
:: A string reflecting the value of the MIME {{ImageDecoderInit/type}} given at
    construction.

: <dfn attribute for=ImageDecoder>\[[complete]]</dfn>
:: A boolean indicating whether {{ImageDecoder/[[encoded data]]}} is completely
    buffered.

: <dfn attribute for=ImageDecoder>[[completed promise]]</dnf>
:: The promise used to signal when {{ImageDecoder/[[complete]]}} becomes
    `true`.

: <dfn attribute for=ImageDecoder>[[codec implementation]]</dfn>
:: An underlying image decoder implementation provided by the User Agent.

: <dfn attribute for=ImageDecoder>[[encoded data]]</dfn>
:: A [=byte sequence=] containing the encoded image data to be decoded.

: <dfn attribute for=ImageDecoder>[[prefer animation]]</dfn>
:: A boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given
    at construction.

: <dfn attribute for=ImageDecoder>[[pending decode promises]]</dfn>
:: A list of unresolved promises returned by calls to decode().

: <dfn attribute for=ImageDecoder>[[internal selected track index]]</dfn>
:: Identifies the image track within {{ImageDecoder/[[encoded data]]}} that is
    used by decoding algorithms on the [=codec thread=].

: <dfn attribute for=ImageDecoder>[[tracks established]]</dfn>
:: A boolean indicating whether the track list has been established in
    {{ImageDecoder/[[ImageTrackList]]}}.

: <dfn attribute for=ImageDecoder>\[[closed]]</dfn>
:: A boolean indicating that the ImageDecoder is in a permanent closed state
    and can no longer be used.

: <dfn attribute for=ImageDecoder>[[progressive frame generations]]</dfn>
:: A mapping of frame indices to [=Progressive Image Frame Generations=]. The
    values represent the Progressive Image Frame Generation for the
    {{VideoFrame}} which was most recently output by a call to
    {{ImageDecoder/decode()}} with the given frame index.


### Constructor ### {#imagedecoder-constructor}

: <dfn constructor for=ImageDecoder title="ImageDecoder(init)">
    ImageDecoder(init)
    </dfn>
:: NOTE: Calling {{ImageDecoder/decode()}} on the constructed {{ImageDecoder}}
    will trigger a {{NotSupportedError}} if the User Agent does not support
    |type|. Authors are encouraged to first check support by calling
    {{ImageDecoder/isTypeSupported()}} with |type|. User Agents don't have to
    support any particular type.

    When invoked, run these steps:
    1. If |init| is not [=valid ImageDecoderInit=], throw a {{TypeError}}.
    2. Let |d| be a new {{ImageDecoder}} object. In the steps below, all
        mentions of {{ImageDecoder}} members apply to |d| unless stated
        otherwise.
    3. Assign {{ImageDecoder/[[ImageTrackList]]}} a new {{ImageTrackList}}
        initialized as follows:
        1. Assign a new [=list=] to {{ImageTrackList/[[track list]]}}.
        2. Assign `-1` to {{ImageTrackList/[[selected index]]}}.
    4. Assign {{ImageDecoderInit/type}} to {{ImageDecoder/[[type]]}}.
    5. Assign `null` to {{ImageDecoder/[[codec implementation]]}}.
    6. If `init.preferAnimation` [=map/exists=], assign `init.preferAnimation`
        to the {{ImageDecoder/[[prefer animation]]}} internal slot. Otherwise,
        assign 'null' to {{ImageDecoder/[[prefer animation]]}} internal slot.
    7. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}.
    8. Assign `-1` to {{ImageDecoder/[[internal selected track index]]}}.
    9. Assign `false` to {{ImageDecoder/[[tracks established]]}}.
    10. Assign `false` to {{ImageDecoder/[[closed]]}}.
    11. Assign a new [=map=] to {{ImageDecoder/[[progressive frame
        generations]]}}.
    12. If |init|'s {{ImageDecoderInit/data}} member is of type
        {{ReadableStream}}:
        1. Assign a new [=list=] to {{ImageDecoder/[[encoded data]]}}.
        2. Assign `false` to {{ImageDecoder/[[complete]]}}
        3. [=Queue a control message=] to [=configure the image decoder=] with
            |init|.
        4. Let |reader| be the result of [=getting a reader=] for
            {{ImageDecoderInit/data}}.
        5. In parallel, perform the [=Fetch Stream Data Loop=] on |d| with
            |reader|.
    13. Otherwise:
        1. Assert that `init.data` is of type {{BufferSource}}.
        2. Assign a copy of `init.data` to  {{ImageDecoder/[[encoded data]]}}.
        3. Assign `true` to {{ImageDecoder/[[complete]]}}.
        4. Reslove {{ImageDecoder/[[completed promise]]}}.
        5. Queue a control message to [=configure the image decoder=] with
            |init|.
        6. Queue a control message to [=decode track metadata=].
    14. return |d|.

    [=Running a control message=] to <dfn>configure the image decoder</dfn>
    means running these steps:
    1. Let |supported| be the result of running the [=ImageDecoder/Check Type
        Support=] algorithm with `init.type`.
    2. If |supported| is `false`, queue a task on the [=control thread=] event
        loop to run the [=ImageDecoder/Close ImageDecoder=] algorithm
        with a {{NotSupportedError}} {{DOMException}} and abort
        these steps.
    3. If |supported| is `true`, assign the
        {{ImageDecoder/[[codec implementation]]}} internal slot with an
        implementation supporting `init.type`
    4. Configure {{ImageDecoder/[[codec implementation]]}} in accordance with
        the values given for {{ImageDecoderInit/premultiplyAlpha}},
        {{ImageDecoderInit/colorSpaceConversion}},
        {{ImageDecoderInit/desiredWidth}}, and
        {{ImageDecoderInit/desiredHeight}}.

    [=Running a control message=] to <dfn>decode track metadata</dfn> means
    running these steps:
    1. Run the [=ImageDecoder/Establish Tracks=] algorithm.

### Attributes ### {#imagedecoder-attributes}
: <dfn attribute for=ImageDecoder>type</dfn>
:: A string reflecting the value of the MIME {{ImageDecoderInit/type}} given at
    construction.

    The {{ImageDecoder/type}} getter steps are to return
    {{ImageDecoder/[[type]]}}.

: <dfn attribute for=ImageDecoder>complete</dfn>
:: Indicates whether {{ImageDecoder/[[encoded data]]}} is completely buffered.

    The {{ImageDecoder/complete}} getter steps are to return
    {{ImageDecoder/[[complete]]}}.

: <dfn attribute for=ImageDecoder>completed</dfn>
:: The promise used to signal when {{ImageDecoder/complete}} becomes `true`.

    The {{ImageDecoder/completed}} getter steps are to return
    {{ImageDecoder/[[completed promise]]}}.

: <dfn attribute for=ImageDecoder>tracks</dfn>
:: Returns a [=live=] {{ImageTrackList}}, which provides metadata
    for the available tracks and a mechanism for selecting a track to decode.

    The {{ImageDecoder/tracks}} getter steps are to return
    {{ImageDecoder/[[ImageTrackList]]}}.

### Methods ### {#imagedecoder-methods}
: <dfn method for=ImageDecoder>decode(options)</dfn>
:: Enqueues a control message to decode the frame according to |options|.

    When invoked, run these steps:
    1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}}
        rejected with an {{InvalidStateError}} {{DOMException}}.
    2. If {{ImageDecoder/[[ImageTrackList]]}}'s
        {{ImageTrackList/[[selected index]]}} is '-1', return a {{Promise}}
        rejected with an {{InvalidStateError}} {{DOMException}}.
    3. If |options| is `undefined`, assign a new {{ImageDecodeOptions}} to
        |options|.
    4. Let |promise| be a new {{Promise}}.
    5. [=Queue a control message=] to decode the image with |options|, and
        |promise|.
    6. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}.
    7. Return |promise|.

    [=Running a control message=] to decode the image means running these
    steps:
    1. Wait for {{ImageDecoder/[[tracks established]]}} to become `true`.
    2. If |options|.{{ImageDecodeOptions/completeFramesOnly}} is `false` and
        the image is a [=Progressive Image=] for which the User Agent supports
        progressive decoding, run the [=Decode Progressive Frame=] algorithm with |options|.{{ImageDecodeOptions/frameIndex}} and |promise|.
    3. Otherwise, run the [=Decode Complete Frame=] algorithm with
        |options|.{{ImageDecodeOptions/frameIndex}} and |promise|.

: <dfn method for=ImageDecoder>reset()</dfn>
:: Immediately aborts all pending work.

    When invoked, run the [=ImageDecoder/Reset ImageDecoder=] algorithm with
    an {{AbortError}} {{DOMException}}.

: <dfn method for=ImageDecoder>close()</dfn>
:: Immediately aborts all pending work and releases system resources. Close is
    final.

    When invoked, run the [=ImageDecoder/Close ImageDecoder=] algorithm with
    an {{AbortError}} {{DOMException}}.

: <dfn method for=ImageDecoder>isTypeSupported(type)</dfn>
:: Returns a promise indicating whether the provided config is supported by the
    User Agent.

    When invoked, run these steps:
    1. If |type| is not a [=valid image MIME type=], return a {{Promise}}
        rejected with {{TypeError}}.
    2. Let |p| be a new {{Promise}}.
    3. In parallel, resolve |p| with the result of running the
        [=Check Type Support=] algorithm with |type|.
    4. Return |p|.

### Algorithms ### {#imagedecoder-algorithms}

: <dfn for=ImageDecoder>Fetch Stream Data Loop</dfn> (with |reader|)
:: Run these steps:
    1. Let |readRequest| be the following [=read request=].

        : [=read request/chunk steps=], given |chunk|
        :: 1. If {{ImageDecoder/[[closed]]}} is `true`, abort these steps.
            2. If |chunk| is not a Uint8Array object, queue a task on the
                [=control thread=] event loop to run the
                [=ImageDecoder/Close ImageDecoder=] algorithm with a
                {{DataError}} {{DOMException}} and abort these steps.
            3. Let |bytes| be the byte sequence represented by the Uint8Array
                object.
            4. Append |bytes| to the  {{ImageDecoder/[[encoded data]]}}
                internal slot.
            5. If {{ImageDecoder/[[tracks established]]}} is `false`, run the
                [=Establish Tracks=] algorithm.
            6. Otherwise, run the [=Update Tracks=] algorithm.
            7. Run the [=Fetch Stream Data Loop=] algorithm with |reader|.

        : [=read request/close steps=]
        :: 1. Assign `true` to {{ImageDecoder/[[complete]]}}
            2. Resolve {{ImageDecoder/[[completed promise]]}}.

        : [=read request/error steps=]
        :: 1. Queue a task on the [=control thread=] event loop to run the
                [=ImageDecoder/Close ImageDecoder=] algorithm with a
                {{NotReadableError}} {{DOMException}}

    2. Read a chunk from |reader| given |readRequest|.

: <dfn for=ImageDecoder>Establish Tracks</dfn>
:: Run these steps:
    1. Assert {{ImageDecoder/[[tracks established]]}} is `false`.
    2. If {{ImageDecoder/[[encoded data]]}} does not contain enough data to
        determine the number of tracks:
        1. If {{ImageDecoder/complete}} is `true`, queue a task on the
            [=control thread=] event loop to run the [=ImageDecoder/Close ImageDecoder=] algorithm.
        2. Abort these steps.
    3. If the number of tracks is found to be `0`, queue a task on the
        [=control thread=] event loop to run the
        [=ImageDecoder/Close ImageDecoder=] algorithm and abort these steps.
    4. Let |newTrackList| be a new [=list=].
    5. For each |image track| found in {{ImageDecoder/[[encoded data]]}}:
        1. Let |newTrack| be a new {{ImageTrack}}, initialized as follows:
            1. Assign [=this=] to {{ImageTrack/[[ImageDecoder]]}}.
            2. Assign {{ImageDecoder/tracks}} to
                {{ImageTrack/[[ImageTrackList]]}}.
            3. If |image track| is found to be animated, assign `true` to
                |newTrack|'s {{ImageTrack/[[animated]]}} internal slot.
                Otherwise, assign `false`.
            4. If |image track| is found to describe a frame count, assign
                that count to |newTrack|'s {{ImageTrack/[[frame count]]}}
                internal slot. Otherwise, assign `0`.

                NOTE: If [=this=] was constructed with
                  {{ImageDecoderInit/data}} as a {{ReadableStream}}, the
                  {{ImageTrack/frameCount}} can change as additional bytes are
                  appended to {{ImageDecoder/[[encoded data]]}}. See the
                  [=Update Tracks=] algorithm.

            5. If |image track| is found to describe a repetition count,
                assign that count to {{ImageTrack/[[repetition count]]}}
                internal slot. Otherwise, assign `0`.

                NOTE: A value of `Infinity` indicates infinite repetitions.

            6. Assign `false` to |newTrack|'s {{ImageTrack/[[selected]]}}
                internal slot.
        2. Append |newTrack| to |newTrackList|.
    6. Let |selectedTrackIndex| be the result of running the
        [=ImageDecoder/Get Default Selected Track Index=] algorithm with
        |newTrackList|.
    7. Let |selectedTrack| be the track at position |selectedTrackIndex| within
        |newTrackList|.
    8. Assign `true` to |selectedTrack|'s {{ImageTrack/[[selected]]}} internal
        slot.
    8. Assign |selectedTrackIndex| to {{ImageDecoder/[[internal selected track
        index]]}}.
    9. Assign `true` to {{ImageDecoder/[[tracks established]]}}.
    10. Queue a task on the [=control thread=] event loop to perform the
        following steps:
        1. Assign |newTrackList| to the {{ImageDecoder/tracks}}
            {{ImageTrackList/[[track list]]}} internal slot.
        2. Assign |selectedTrackIndex| to {{ImageDecoder/tracks}}
            {{ImageTrackList/[[selected index]]}}.
        3. Resolve {{ImageTrackList/[[ready promise]]}}.

: <dfn for=ImageDecoder>Get Default Selected Track Index</dfn> (with
    |trackList|)
:: Run these steps:
    1. If {{ImageDecoder/[[encoded data]]}} identifies a [=Primary Image
        Track=]:
        1. Let |primaryTrack| be the {{ImageTrack}} from |trackList| that
            describes the [=Primary Image Track=].
        2. Let |primaryTrackIndex| be position of |primaryTrack| within
            |trackList|.
        3. If {{ImageDecoder/[[prefer animation]]}} is `null`, return
            |primaryTrackIndex|.
        4. If |primaryTrack|.{{ImageTrack/animated}} equals
            {{ImageDecoder/[[prefer animation]]}}, return |primaryTrackIndex|.
    2. If any {{ImageTrack}}s in |trackList| have {{ImageTrack/animated}} equal
        to {{ImageDecoder/[[prefer animation]]}}, return the position of the
        earliest such track in |trackList|.
    3. Return `0`.

: <dfn for=ImageDecoder>Update Tracks</dfn>
:: A <dfn>track update struct</dfn> is a [=struct=] that consists of a
    <dfn for="track update struct">track index</dfn> ({{unsigned long}})
    and a <dfn for="track update struct">frame count</dfn>
    ({{unsigned long}}).

    Run these steps:
    1. Assert {{ImageDecoder/[[tracks established]]}} is `true`.
    2. Let |trackChanges| be a new [=list=].
    3. Let |trackList| be a copy of {{ImageDecoder/tracks}}'
        {{ImageTrackList/[[track list]]}}.
    4. For each |track| in |trackList|:
        1. Let |trackIndex| be  the position of |track| in |trackList|.
        2. Let |latestFrameCount| be the frame count as indicated by
            {{ImageDecoder/[[encoded data]]}} for the track corresponding to
            |track|.
        3. Assert that |latestFrameCount| is greater than or equal to
            `track.frameCount`.
        4. If |latestFrameCount| is greater than `track.frameCount`:
            1. Let |change| be a [=track update struct=] whose
                [=track update struct/track index=] is |trackIndex| and
                [=track update struct/frame count=] is |latestFrameCount|.
            2. Append |change| to |tracksChanges|.
    5. If |tracksChanges| is [=list/empty=], abort these steps.
    6. Queue a task on the [=control thread=] event loop to perform the
        following steps:
        1. For each <var ignore=''>update</var> in |trackChanges|:
            1. Let |updateTrack| be the {{ImageTrack}} at position
                `update.trackIndex` within {{ImageDecoder/tracks}}'
                {{ImageTrackList/[[track list]]}}.
            2. Assign `update.frameCount` to |updateTrack|'s
                {{ImageTrack/[[frame count]]}}.
            3. Fire a simple event named {{ImageTrack/change}} at the
                {{ImageDecoder/tracks}} object.

: <dfn for=ImageDecoder>Decode Complete Frame</dfn> (with |frameIndex| and
    |promise|)
:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`.
    2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not
        `-1`.
    3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and
        {{ImageDecoder/[[internal selected track index]]}}.
    4. Wait for any of the following conditions to be true (whichever happens
        first):
        1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to
            completely decode |encodedFrame|.
        2. {{ImageDecoder/[[encoded data]]}} is found to be malformed.
        3. {{ImageDecoder/complete}} is `true`.
        4. {{ImageDecoder/[[closed]]}} is `true`.
    5. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    6. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to
        completely decode |encodedFrame|, run the
        [=ImageDecoder/Reject Infeasible Decode=] algorithm with |promise| and
        abort these steps.
    7. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
        |encodedFrame|.
    8. If decoding produces an error, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    9. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry
        keyed by |frameIndex|, remove the entry from the map.
    10. Let |output| be the decoded image data emitted by
        {{ImageDecoder/[[codec implementation]]}} corresponding to
        |encodedFrame|.
    11. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as
        follows:
        1. Assign 'true' to {{ImageDecodeResult/complete}}.
        2. Let |timestamp| and |duration| be the presentation timestamp and
            duration for |output| as described by |encodedFrame|. If
            |encodedFrame| does not describe a timestamp or
            duration, assign `null` to the corresponding variable.
        3. Assign {{ImageDecodeResult/image}} with the result of running the
            [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
            |duration|.
    12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
        |decodeResult|.

: <dfn for=ImageDecoder>Decode Progressive Frame</dfn> (with |frameIndex| and
    |promise|)
:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`.
    2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not
        `-1`.
    3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and
        {{ImageDecoder/[[internal selected track index]]}}.
    4. Let |lastFrameGeneration| be `null`.
    5. If {{ImageDecoder/[[progressive frame generations]]}} contains a map
        entry with the key |frameIndex|, assign the value of the map entry to
        |lastFrameGeneration|.
    6. Wait for any of the following conditions to be true (whichever happens
        first):
        1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to decode
            |encodedFrame| to produce an output whose [=Progressive Image
            Frame Generation=] exceeds |lastFrameGeneration|.
        2. {{ImageDecoder/[[encoded data]]}} is found to be malformed.
        3. {{ImageDecoder/complete}} is `true`.
        4. {{ImageDecoder/[[closed]]}} is `true`.
    7. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    8. Otherwise, if {{ImageDecoder/[[encoded data]]}} does not contain enough
        bytes to decode |encodedFrame| to produce an output whose
        [=Progressive Image Frame Generation=] exceeds |lastFrameGeneration|,
        run the [=ImageDecoder/Reject Infeasible Decode=] algorithm with
        |promise| and abort these steps.
    9. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
        |encodedFrame|.
    10. If decoding produces an error, run the
        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
        steps.
    11. Let |output| be the decoded image data emitted by
        {{ImageDecoder/[[codec implementation]]}} corresponding to
        |encodedFrame|.
    12. Let |decodeResult| be a new {{ImageDecodeResult}}.
    13. If |output| is the final full-detail progressive output corresponding
        to |encodedFrame|:
        1. Assign `true` to |decodeResult|'s {{ImageDecodeResult/complete}}.
        2. If {{ImageDecoder/[[progressive frame generations]]}} contains an
            entry keyed by |frameIndex|, remove the entry from the map.
    14. Otherwise:
        1. Assign `false` to |decodeResult|'s {{ImageDecodeResult/complete}}.
        2. Let |frameGeneration| be the [=Progressive Image Frame Generation=]
            for |output|.
        3. Add a new entry to {{ImageDecoder/[[progressive frame
            generations]]}} with key |frameIndex| and value |frameGeneration|.
    15. Let |timestamp| and |duration| be the presentation timestamp and
            duration for |output| as described by |encodedFrame|. If
            |encodedFrame| does not describe a timestamp or
            duration, assign `null` to the corresponding variable.
    16. Assign {{ImageDecodeResult/image}} with the result of running the
            [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
            |duration|.
    17. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
    18. Resolve |promise| with |decodeResult|.

: <dfn for=ImageDecoder>Resolve Decode</dfn> (with |promise| and |result|)
:: 1. Queue a task on the [=control thread=] event loop to run these steps:
        1. If {{ImageDecoder/[[closed]]}}, abort these steps.
        2. Assert that |promise| is an element of
            {{ImageDecoder/[[pending decode promises]]}}.
        3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
        4. Resolve |promise| with |result|.

: <dfn for=ImageDecoder>Reject Infeasible Decode</dfn> (with |promise|)
:: 1. Assert that {{ImageDecoder/complete}} is `true` or
        {{ImageDecoder/[[closed]]}} is `true`.
    2. If {{ImageDecoder/complete}} is `true`, let |exception| be a
            {{RangeError}}. Otherwise, let |exception| be an
            {{InvalidStateError}} {{DOMException}}.
    3. Queue a task on the [=control thread=] event loop to run these steps:
        1. If {{ImageDecoder/[[closed]]}}, abort these steps.
        2. Assert that |promise| is an element of
            {{ImageDecoder/[[pending decode promises]]}}.
        3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
        4. Reject |promise| with |exception|.

: <dfn for=ImageDecoder>Fatally Reject Bad Data</dfn>
:: 1. Queue a task on the [=control thread=] event loop to run these steps:
        1. If {{ImageDecoder/[[closed]]}}, abort these steps.
        2. Run the [=ImageDecoder/Close ImageDecoder=] algorithm with an
            {{EncodingError}} {{DOMException}}.

: <dfn for=ImageDecoder>Check Type Support</dfn> (with |type|)
:: 1. If the User Agent can provide a codec to support decoding |type|, return
        `true`.
    2. Otherwise, return `false`.

: <dfn for=ImageDecoder>Reset ImageDecoder</dfn> (with |exception|)
:: 1. Signal {{ImageDecoder/[[codec implementation]]}} to abort any active
        decoding operation.
    2. For each |decodePromise| in
        {{ImageDecoder/[[pending decode promises]]}}:
        1. Reject |decodePromise| with |exception|.
        2. Remove |decodePromise| from
            {{ImageDecoder/[[pending decode promises]]}}.

: <dfn for=ImageDecoder>Close ImageDecoder</dfn> (with |exception|)
:: 1. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm with |exception|.
    1. Assign `true` to {{ImageDecoder/[[closed]]}}.
    2. Clear {{ImageDecoder/[[codec implementation]]}} and release associated
        [=system resources=].
    3. Remove all entries from {{ImageDecoder/[[ImageTrackList]]}}.
    4. Assign `-1` to {{ImageDecoder/[[ImageTrackList]]}}'s
        {{ImageTrackList/[[selected index]]}}.


ImageDecoderInit Interface {#imagedecoderinit-interface}
--------------------------------------------------------
<pre class='idl'>
<xmp>
typedef (BufferSource or ReadableStream) ImageBufferSource;
dictionary ImageDecoderInit {
  required DOMString type;
  required ImageBufferSource data;
  PremultiplyAlpha premultiplyAlpha = "default";
  ColorSpaceConversion colorSpaceConversion = "default";
  [EnforceRange] unsigned long desiredWidth;
  [EnforceRange] unsigned long desiredHeight;
  boolean preferAnimation;
};
</xmp>
</pre>

To determine if an {{ImageDecoderInit}} is a <dfn>valid ImageDecoderInit</dfn>,
run these steps:
1. If |type| is not a [=valid image MIME type=], return `false`.
2. If |data| is of type {{ReadableStream}} and the ReadableStream is
    [=ReadableStream/disturbed=] or [=ReadableStream/locked=], return `false`.
3. If |data| is of type {{BufferSource}}:
    1. If the result of running  IsDetachedBuffer (described in
        [[!ECMASCRIPT]]) on |data| is `false`, return `false`.
    2. If |data| is [=empty=], return `false`.
4. If {{ImageDecoderInit/desiredWidth}} [=map/exists=] and
    {{ImageDecoderInit/desiredHeight}} does not exist, return `false`.
5. If {{ImageDecoderInit/desiredHeight}} [=map/exists=] and
    {{ImageDecoderInit/desiredWidth}} does not exist, return `false`.
6. Return `true`.

A <dfn>valid image MIME type</dfn> is a string that is a [=valid MIME type
string=] and for which the `type`, per Section 3.1.1.1 of [[RFC7231]], is
`image`.

: <dfn dict-member for=ImageDecoderInit>type</dfn>
:: String containing the MIME type of the image file to be decoded.

: <dfn dict-member for=ImageDecoderInit>data</dfn>
:: {{BufferSource}} or {{ReadableStream}} of bytes representing an encoded
    image file as described by {{ImageDecoderInit/type}}.

: <dfn dict-member for=ImageDecoderInit>premultiplyAlpha</dfn>
:: Controls whether decoded outputs' color channels are to be premultiplied by
    their alpha channel, as defined by {{ImageBitmapOptions/premultiplyAlpha}}
    in {{ImageBitmapOptions}}.

: <dfn dict-member for=ImageDecoderInit>colorSpaceConversion</dfn>
:: Controls whether decoded outputs' color space is converted or ignored, as
    defined by {{ImageBitmapOptions/colorSpaceConversion}} in
    {{ImageBitmapOptions}}.

: <dfn dict-member for=ImageDecoderInit>desiredWidth</dfn>
:: Indicates a desired width for decoded outputs. Implementation is best
    effort; decoding to a desired width <em class="rfc2119">MAY</em> not be
    supported by all formats/ decoders.

: <dfn dict-member for=ImageDecoderInit>desiredHeight</dfn>
:: Indicates a desired height for decoded outputs. Implementation is best
    effort; decoding to a desired height <em class="rfc2119">MAY</em> not be
    supported by all formats/decoders.

: <dfn dict-member for=ImageDecoderInit>preferAnimation</dfn>
:: For images with multiple tracks, this indicates whether the
    initial track selection <em class="rfc2119">SHOULD</em> prefer an animated
    track.

    NOTE: See the [=ImageDecoder/Get Default Selected Track Index=] algorithm.

ImageDecodeOptions Interface {#imagedecodeoptions-interface}
------------------------------------------------------------
<pre class='idl'>
<xmp>
dictionary ImageDecodeOptions {
  [EnforceRange] unsigned long frameIndex = 0;
  boolean completeFramesOnly = true;
};
</xmp>
</pre>

: <dfn dict-member for=ImageDecodeOptions>frameIndex</dfn>
:: The index of the frame to decode.

: <dfn dict-member for=ImageDecodeOptions>completeFramesOnly</dfn>
:: For [=Progressive Images=], a value of `false` indicates that the decoder
    <em class="rfc2119">MAY</em> output an {{ImageDecodeResult/image}} with
    reduced detail. Each subsequent call to {{ImageDecoder/decode()}} for the
    same {{ImageDecodeOptions/frameIndex}} will resolve to produce an image with
    a higher [=Progressive Image Frame Generation=] (more image detail) than the
    previous call, until finally the full-detail image is produced.

    If {{ImageDecodeOptions/completeFramesOnly}} is assigned `true`, or if the
    image is not a [=Progressive Image=], or if the User Agent does not support
    progressive decoding for the given image type, calls to
    {{ImageDecoder/decode()}} will only resolve once the full detail image is
    decoded.

    <div class='note'>
      NOTE: For [=Progressive Images=], setting
          {{ImageDecodeOptions/completeFramesOnly}} to `false` can be used to
          offer users a preview an image that is still being buffered from the
          network (via the {{ImageDecoderInit/data}} {{ReadableStream}}).

          Upon decoding the full detail image, the {{ImageDecodeResult}}'s
          {{ImageDecodeResult/complete}} will be set to true.
    </div>


ImageDecodeResult Interface {#imagedecoderesult-interface}
----------------------------------------------------------
<pre class='idl'>
<xmp>
dictionary ImageDecodeResult {
  required VideoFrame image;
  required boolean complete;
};
</xmp>
</pre>

: <dfn dict-member for=ImageDecodeResult>image</dfn>
:: The decoded image.

: <dfn dict-member for=ImageDecodeResult>complete</dfn>
:: Indicates whether {{ImageDecodeResult/image}} contains the final full-detail
    output.

    NOTE: {{ImageDecodeResult/complete}} is always `true` when
        {{ImageDecoder/decode()}} is invoked with
        {{ImageDecodeOptions/completeFramesOnly}} set to `true`.

ImageTrackList Interface {#imagetracklist-interface}
----------------------------------------------------
<pre class='idl'>
<xmp>
[Exposed=(Window,DedicatedWorker)]
interface ImageTrackList {
  getter ImageTrack (unsigned long index);

  readonly attribute Promise<undefined> ready;
  readonly attribute unsigned long length;
  readonly attribute long selectedIndex;
  readonly attribute ImageTrack? selectedTrack;
};
</xmp>
</pre>

### Internal Slots ### {#imagetracklist-internal-slots}
: <dfn attribute for=ImageTrackList>[[ready promise]]</dfn>
:: The promise used to signal when the {{ImageTrackList}} has been populated
    with {{ImageTrack}}s.

    NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} can receive subsequent
        updates until {{ImageDecoder/complete}} is `true`.

: <dfn attribute for=ImageTrackList>[[track list]]</dfn>
:: The list of {{ImageTrack}}s describe by this {{ImageTrackList}}.

: <dfn attribute for=ImageTrackList>\[[selected index]]</dfn>
:: The index of the selected track in {{ImageTrackList/[[track list]]}}. A
    value of `-1` indicates that no track is selected.

### Attributes ### {#imagetracklist-attributes}
: <dfn attribute for=ImageTrackList>ready</dfn>
:: The {{ImageTrackList/ready}} getter steps are to return the
    {{ImageTrackList/[[ready promise]]}}.

: <dfn attribute for=ImageTrackList>length</dfn>
:: The {{ImageTrackList/length}} getter steps are to return the length of
    {{ImageTrackList/[[track list]]}}.

: <dfn attribute for=ImageTrackList>selectedIndex</dfn>
:: The {{ImageTrackList/selectedIndex}} getter steps are to return
    {{ImageTrackList/[[selected index]]}};

: <dfn attribute for=ImageTrackList>selectedTrack</dfn>
:: The {{ImageTrackList/selectedTrack}} getter steps are:
    1. If {{ImageTrackList/[[selected index]]}} is `-1`, return `null`.
    2. Otherwise, return the ImageTrack from {{ImageTrackList/[[track list]]}}
        at the position indicated by {{ImageTrackList/[[selected index]]}}.

ImageTrack Interface {#imagetrack-interface}
--------------------------------------------
<pre class='idl'>
<xmp>
[Exposed=(Window,DedicatedWorker)]
interface ImageTrack : EventTarget {
  readonly attribute boolean animated;
  readonly attribute unsigned long frameCount;
  readonly attribute unrestricted float repetitionCount;
  attribute EventHandler onchange;
  attribute boolean selected;
};
</xmp>
</pre>

### Internal Slots ### {#imagetrack-internal-slots}
: <dfn attribute for=ImageTrack>\[[ImageDecoder]]</dfn>
:: The {{ImageDecoder}} instance that constructed this {{ImageTrack}}.

: <dfn attribute for=ImageTrack>\[[ImageTrackList]]</dfn>
:: The {{ImageTrackList}} instance that lists this {{ImageTrack}}.

: <dfn attribute for=ImageTrack>\[[animated]]</dfn>
:: Indicates whether this track contains an animated image with multiple
    frames.

: <dfn attribute for=ImageTrack>[[frame count]]</dfn>
:: The number of frames in this track.

: <dfn attribute for=ImageTrack>[[repetition count]]</dfn>
:: The number of times the animation is intended to repeat.

: <dfn attribute for=ImageTrack>\[[selected]]</dfn>
:: Indicates whether this track is selected for decoding.

### Attributes ### {#imagetrack-attributes}

: <dfn attribute for=ImageTrack>animated</dfn>
:: The {{ImageTrack/animated}} getter steps are to return the value of
    {{ImageTrack/[[animated]]}}.

    NOTE: This attribute provides an early indication that
        {{ImageTrack/frameCount}} will ultimately exceed 0 for images where the
        {{ImageTrack/frameCount}} starts at `0` and later increments as new
        chunks of the {{ReadableStream}} {{ImageDecoderInit/data}} arrive.

: <dfn attribute for=ImageTrack>frameCount</dfn>
:: The {{ImageTrack/frameCount}} getter steps are to return the value of
    {{ImageTrack/[[frame count]]}}.

: <dfn attribute for=ImageTrack>repetitionCount</dfn>
:: The {{ImageTrack/repetitionCount}} getter steps are to return the value of
    {{ImageTrack/[[repetition count]]}}.

: <dfn attribute for=ImageTrack>onchange</dfn>
:: An [=event handler IDL attribute=] whose [=event handler event type=] is
    {{ImageTrack/change}}.

: <dfn attribute for=ImageTrack>selected</dfn>
:: The {{ImageTrack/selected}} getter steps are to return the value of
    {{ImageTrack/[[selected]]}}.

    The {{ImageTrack/selected}} setter steps are:
    1. If {{ImageTrack/[[ImageDecoder]]}}'s {{ImageDecoder/[[closed]]}} slot is
        `true`, abort these steps.
    2. Let |newValue| be [=the given value=].
    3. If |newValue| equals {{ImageTrack/[[selected]]}}, abort these steps.
    4. Assign |newValue| to {{ImageTrack/[[selected]]}}.
    5. Let |parentTrackList| be {{ImageTrack/[[ImageTrackList]]}}
    6. Let |oldSelectedIndex| be the value of |parentTrackList|
        {{ImageTrackList/[[selected index]]}}.
    7. If |oldSelectedIndex| is not `-1`:
        1. Let |oldSelectedTrack| be the {{ImageTrack}} in |parentTrackList|
            {{ImageTrackList/[[track list]]}} at the position of
            |oldSelectedIndex|.
        2. Assign `false` to |oldSelectedTrack| {{ImageTrack/[[selected]]}}

    8. If |newValue| is `true`, let |selectedIndex| be the index of [=this=]
        {{ImageTrack}} within |parentTrackList|'s
        {{ImageTrackList/[[track list]]}}. Otherwise, let |selectedIndex| be
        `-1`.
    9. Assign |selectedIndex| to |parentTrackList|
        {{ImageTrackList/[[selected index]]}}.
    10. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm on
        {{ImageTrack/[[ImageDecoder]]}}.
    11. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s
        [=control message queue=] to update the internal selected track
        index with |selectedIndex|.

    [=Running a control message=] to update the internal selected track index
    means running these steps:
    1. Assign |selectedIndex| to
        {{ImageDecoder/[[internal selected track index]]}}.
    2. Remove all entries from
        {{ImageDecoder/[[progressive frame generations]]}}.


### Event Summary ### {#imagetracklist-eventsummary}

: <dfn event for=ImageTrack>change</dfn>
:: Fired at the {{ImageTrack}} when the {{ImageTrack/frameCount}} is altered.

Resource Reclamation{#resource-reclamation}
==============================================

When resources are constrained, a User Agent MAY proactively reclaim codecs.
This is particularly true in the case where hardware codecs are limited, and
shared accross web pages or platform apps.

To <dfn>reclaim a codec</dfn>, a User Agent MUST run the appropriate
close algorithm (amongst [=Close AudioDecoder=], [=Close AudioEncoder=],
[=Close VideoDecoder=] and [=Close VideoEncoder=]) with a {{QuotaExceededError}}
{{DOMException}}.

The rules governing when a codec may be reclaimed depend on whether the codec is
an [=active=] or [=inactive=] codec and/or a [=background=] codec.

An <dfn lt="active codec|active">active codec</dfn> is a codec that has
received a call to `encode()`, `decode()`, `configure()`, `flush()` or `reset()`
in the past `10 seconds`, or has called its `output()` callback in the past `10
seconds`. Addionally, {{VideoEncoder}}s are considered [=active=] if they are
making progress in encoding queued {{VideoFrame}}s.

NOTE: Encoding large {{VideoFrame}}s can take more than `10s` per frame. The
special case for {{VideoEncoder}}s ensures that they are not reclaimed if more
than `10 seconds` elapses between each `output()` callback.

An <dfn lt="inactive codec|inactive">inactive codec</dfn> is any codec that does
not meet the definition of an [=active codec=].

A <dfn lt="background codec|background">background codec</dfn> is a codec whose
{{ownerDocument}} (or [=owner set=]'s {{Document}}, for codecs in workers) has a
{{Document/hidden}} attribute equal to `true`.

A User Agent MUST only [=reclaim a codec=] that is either an
[=inactive codec=], a [=background codec=], or both. A User Agent MUST NOT
reclaim a codec that is both [=active=] and in the foreground, i.e. not a
[=background codec=].

Additionally, User Agents MUST NOT reclaim an [=active=] [=background=] codec
if it is:
    - An encoder, e.g. an {{AudioEncoder}} or {{VideoEncoder}}.

        NOTE: This prevents long running encode tasks from being interrupted.

    - An {{AudioDecoder}} or {{VideoDecoder}}, when there is, respectively, an
        [=active=] {{AudioEncoder}} or {{VideoEncoder}} in the same
        [=webappapis/global object=].

        NOTE: This prevents prevents breaking long running transcoding tasks.

    - An {{AudioDecoder}}, when its tab is audibly playing audio.

Security Considerations{#security-considerations}
=================================================

The primary security impact is that features of this API make it easier for an
attacker to exploit vulnerabilities in the underlying platform codecs.
Additionally, new abilities to configure and control the codecs <em
class="rfc2119">MAY</em> allow for new exploits that rely on a specific
configuration and/or sequence of control operations.

Platform codecs are historically an internal detail of APIs like
{{HTMLMediaElement}}, [[WEBAUDIO]], and [[WebRTC]]. In this way, it has always
been possible to attack the underlying codecs by using malformed media
files/streams and invoking the various API control methods.

For example, you can send any stream to a decoder by first wrapping that stream
in a media container (e.g. mp4) and setting that as the {{HTMLMediaElement/src}}
of an {{HTMLMediaElement}}. You can then cause the underlying video decoder to
be {{VideoDecoder/reset()}} by setting a new value for `<video>.currentTime`.

WebCodecs makes such attacks easier by exposing low level control when inputs
are provided and direct access to invoke the codec control methods. This also
affords attackers the ability to invoke sequences of control methods that were
not previously possible via the higher level APIs.

User Agents <em class="rfc2119">SHOULD</em> mitigate this risk by extensively
fuzzing their implementation with random inputs and control method invocations.
Additionally, User Agents are encouraged to isolate their underlying codecs in
processes with restricted privileges (sandbox) as a barrier against successful
exploits being able to read user data.

An additional concern is exposing the underlying codecs to input mutation race
conditions. Specifically, it <em class="rfc2119">SHOULD</em> not be possible for
a site to mutate a codec input or output while the underlying codec <em
class="rfc2119">MAY</em> still be operating on that data.  This concern is
mitigated by ensuring that input and output interfaces are immutable.

Privacy Considerations{#privacy-considerations}
===============================================

The primary privacy impact is an increased ability to fingerprint users by
querying for different codec capabilities to establish a codec feature profile.
Much of this profile is already exposed by existing APIs. Such profiles are very
unlikely to be uniquely identifying, but <em class="rfc2119">MAY</em> be used
with other metrics to create a fingerprint.

An attacker <em class="rfc2119">MAY</em> accumulate a codec feature profile by
calling `IsConfigSupported()` methods with a number of different configuration
dictionaries. Similarly, an attacker <em class="rfc2119">MAY</em> attempt to
`configure()` a codec with different configuration dictionaries and observe
which configurations are accepted.

Attackers <em class="rfc2119">MAY</em> also use existing APIs to establish much
of the codec feature profile. For example, the [[media-capabilities]]
{{decodingInfo()}} API describes what types of decoders are supported and its
{{powerEfficient}} attribute <em class="rfc2119">MAY</em> signal when a decoder
uses hardware acceleration. Similarly, the [[WebRTC]]
{{RTCRtpSender/getCapabilities()}} API <em class="rfc2119">MAY</em> be used to
determine what types of encoders are supported and the
{{RTCPeerConnection/getStats()}} API <em class="rfc2119">MAY</em> be used to
determine when an encoder uses hardware acceleration. WebCodecs will expose some
additional information in the form of low level codec features.

A codec feature profile alone is unlikely to be uniquely identifying. Underlying
codecs are often implemented entirely in software (be it part of the User Agent
binary or part of the operating system), such that all users who run that
software will have a common set of capabilities. Additionally, underlying codecs
are often implemented with hardware acceleration, but such hardware is mass
produced and devices of a particular class and manufacture date (e.g. flagship
phones manufactured in 2020) will often have common capabilities. There will be
outliers (some users <em class="rfc2119">MAY</em> run outdated versions of
software codecs or use a rare mix of custom assembled hardware), but most of the
time a given codec feature profile is shared by a large group of users.

Segmenting groups of users by codec feature profile still amounts to a bit of
entropy that can be combined with other metrics to uniquely identify a user.
User Agents <em class="rfc2119">MAY</em> partially mitigate this by returning an
error whenever a site attempts to exhaustively probe for codec capabilities.
Additionally, User Agents <em class="rfc2119">MAY</em> implement a "privacy
budget", which depletes as authors use WebCodecs and other identifying APIs.
Upon exhaustion of the privacy budget, codec capabilities could be reduced to a
common baseline or prompt for user approval.

Best Practices for Authors Using WebCodecs{#best-practices-developers}
======================================================================

<div class=non-normative>
  This section is non-normative.

  While WebCodecs internally operates on background threads, authors working with
  realtime media or in contended main thread environments are encouraged to ensure their
  media pipelines operate in worker contexts entirely independent of the main
  thread where possible. For example, realtime media processing of {{VideoFrame}}s
  are generally be done in a worker context.

  The main thread has significant potential for high contention and jank that can
  go unnoticed in development, yet degrade inconsistently across devices and User
  Agents in the field -- potentially dramatically impacting the end user
  experience. Ensuring the media pipeline is decoupled from the main thread helps
  provide a smooth experience for end users.

  Authors using the main thread for their media pipeline ought to be sure of
  their target frame rates, main thread workload, how their application will be
  embedded, and the class of devices their users will be using.
</div>