Android APIs
public final class

MediaCodec

extends Object
java.lang.Object
   ↳ android.media.MediaCodec

Class Overview

MediaCodec class can be used to access low-level media codec, i.e. encoder/decoder components.

MediaCodec is generally used like this:

 MediaCodec codec = MediaCodec.createDecoderByType(type);
 codec.configure(format, ...);
 codec.start();
 ByteBuffer[] inputBuffers = codec.getInputBuffers();
 ByteBuffer[] outputBuffers = codec.getOutputBuffers();
 for (;;) {
   int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs);
   if (inputBufferIndex >= 0) {
     // fill inputBuffers[inputBufferIndex] with valid data
     ...
     codec.queueInputBuffer(inputBufferIndex, ...);
   }

   int outputBufferIndex = codec.dequeueOutputBuffer(timeoutUs);
   if (outputBufferIndex >= 0) {
     // outputBuffer is ready to be processed or rendered.
     ...
     codec.releaseOutputBuffer(outputBufferIndex, ...);
   } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
     outputBuffers = codec.getOutputBuffers();
   } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
     // Subsequent data will conform to new format.
     MediaFormat format = codec.getOutputFormat();
     ...
   }
 }
 codec.stop();
 codec.release();
 codec = null;
 
Each codec maintains a number of input and output buffers that are referred to by index in API calls. The contents of these buffers are represented by the ByteBuffer[] arrays accessible through getInputBuffers() and getOutputBuffers(). After a successful call to start() the client "owns" neither input nor output buffers, subsequent calls to dequeueInputBuffer(long) and dequeueOutputBuffer(MediaCodec.BufferInfo, long) then transfer ownership from the codec to the client.

The client is not required to resubmit/release buffers immediately to the codec, the sample code above simply does this for simplicity's sake.

Once the client has an input buffer available it can fill it with data and submit it it to the codec via a call to queueInputBuffer(int, int, int, long, int).

The codec in turn will return an output buffer to the client in response to dequeueOutputBuffer(MediaCodec.BufferInfo, long). After the output buffer has been processed a call to releaseOutputBuffer(int, boolean) will return it to the codec. If a video surface has been provided in the call to configure(MediaFormat, Surface, MediaCrypto, int), releaseOutputBuffer(int, boolean) optionally allows rendering of the buffer to the surface.

Input buffers (for decoders) and Output buffers (for encoders) contain encoded data according to the format's type. For video types this data is all the encoded data representing a single moment in time, for audio data this is slightly relaxed in that a buffer may contain multiple encoded frames of audio. In either case, buffers do not start and end on arbitrary byte boundaries, this is not a stream of bytes, it's a stream of access units.

Most formats also require the actual data to be prefixed by a number of buffers containing setup data, or codec specific data, i.e. the first few buffers submitted to the codec object after starting it must be codec specific data marked as such using the flag BUFFER_FLAG_CODEC_CONFIG in a call to queueInputBuffer(int, int, int, long, int). Codec specific data included in the format passed to configure(MediaFormat, Surface, MediaCrypto, int) (in ByteBuffer entries with keys "csd-0", "csd-1", ...) is automatically submitted to the codec, this data MUST NOT be submitted explicitly by the client. Once the client reaches the end of the input data it signals the end of the input stream by specifying a flag of BUFFER_FLAG_END_OF_STREAM in the call to queueInputBuffer(int, int, int, long, int). The codec will continue to return output buffers until it eventually signals the end of the output stream by specifying the same flag (BUFFER_FLAG_END_OF_STREAM) on the BufferInfo returned in dequeueOutputBuffer(MediaCodec.BufferInfo, long). In order to start decoding data that's not adjacent to previously submitted data (i.e. after a seek) it is necessary to flush() the decoder. Any input or output buffers the client may own at the point of the flush are immediately revoked, i.e. after a call to flush() the client does not own any buffers anymore. Note that the format of the data submitted after a flush must not change, flush does not support format discontinuities, for this a full stop(), configure(MediaFormat, Surface, MediaCrypto, int), start() cycle is necessary.

The factory methods createByCodecName(String), createDecoderByType(String), and createEncoderByType(String) throw IOException on failure which the caller must catch or declare to pass up. Other methods will throw IllegalStateException if the codec is in an Uninitialized, Invalid, or Error state (e.g. not initialized properly). Exceptions are thrown elsewhere as noted.

Summary

Nested Classes
class MediaCodec.BufferInfo Per buffer metadata includes an offset and size specifying the range of valid data in the associated codec buffer. 
class MediaCodec.CryptoException Thrown when a crypto error occurs while queueing a secure input buffer. 
class MediaCodec.CryptoInfo Metadata describing the structure of a (at least partially) encrypted input sample. 
Constants
int BUFFER_FLAG_CODEC_CONFIG This indicated that the buffer marked as such contains codec initialization / codec specific data instead of media data.
int BUFFER_FLAG_END_OF_STREAM This signals the end of stream, i.e.
int BUFFER_FLAG_SYNC_FRAME This indicates that the buffer marked as such contains the data for a sync frame.
int CONFIGURE_FLAG_ENCODE If this codec is to be used as an encoder, pass this flag.
int CRYPTO_MODE_AES_CTR
int CRYPTO_MODE_UNENCRYPTED
int INFO_OUTPUT_BUFFERS_CHANGED The output buffers have changed, the client must refer to the new set of output buffers returned by getOutputBuffers() from this point on.
int INFO_OUTPUT_FORMAT_CHANGED The output format has changed, subsequent data will follow the new format.
int INFO_TRY_AGAIN_LATER If a non-negative timeout had been specified in the call to dequeueOutputBuffer(MediaCodec.BufferInfo, long), indicates that the call timed out.
String PARAMETER_KEY_REQUEST_SYNC_FRAME Request that the encoder produce a sync frame "soon".
String PARAMETER_KEY_SUSPEND Temporarily suspend/resume encoding of input data.
String PARAMETER_KEY_VIDEO_BITRATE Change a video encoder's target bitrate on the fly.
int VIDEO_SCALING_MODE_SCALE_TO_FIT The content is scaled to the surface dimensions
int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING The content is scaled, maintaining its aspect ratio, the whole surface area is used, content may be cropped
Public Methods
void configure(MediaFormat format, Surface surface, MediaCrypto crypto, int flags)
Configures a component.
static MediaCodec createByCodecName(String name)
If you know the exact name of the component you want to instantiate use this method to instantiate it.
static MediaCodec createDecoderByType(String type)
Instantiate a decoder supporting input data of the given mime type.
static MediaCodec createEncoderByType(String type)
Instantiate an encoder supporting output data of the given mime type.
final Surface createInputSurface()
Requests a Surface to use as the input to an encoder, in place of input buffers.
final int dequeueInputBuffer(long timeoutUs)
Returns the index of an input buffer to be filled with valid data or -1 if no such buffer is currently available.
final int dequeueOutputBuffer(MediaCodec.BufferInfo info, long timeoutUs)
Dequeue an output buffer, block at most "timeoutUs" microseconds.
final void flush()
Flush both input and output ports of the component, all indices previously returned in calls to dequeueInputBuffer(long) and dequeueOutputBuffer(MediaCodec.BufferInfo, long) become invalid.
MediaCodecInfo getCodecInfo()
Get the codec info.
ByteBuffer[] getInputBuffers()
Call this after start() returns.
final String getName()
Get the component name.
ByteBuffer[] getOutputBuffers()
Call this after start() returns and whenever dequeueOutputBuffer signals an output buffer change by returning INFO_OUTPUT_BUFFERS_CHANGED
final MediaFormat getOutputFormat()
Call this after dequeueOutputBuffer signals a format change by returning INFO_OUTPUT_FORMAT_CHANGED
final void queueInputBuffer(int index, int offset, int size, long presentationTimeUs, int flags)
After filling a range of the input buffer at the specified index submit it to the component.
final void queueSecureInputBuffer(int index, int offset, MediaCodec.CryptoInfo info, long presentationTimeUs, int flags)
Similar to queueInputBuffer(int, int, int, long, int) but submits a buffer that is potentially encrypted.
final void release()
Make sure you call this when you're done to free up any opened component instance instead of relying on the garbage collector to do this for you at some point in the future.
final void releaseOutputBuffer(int index, boolean render)
If you are done with a buffer, use this call to return the buffer to the codec.
final void releaseOutputBuffer(int index, long renderTimestampNs)
If you are done with a buffer, use this call to update its surface timestamp and return it to the codec to render it on the output surface.
final void setParameters(Bundle params)
Communicate additional parameter changes to the component instance.
final void setVideoScalingMode(int mode)
If a surface has been specified in a previous call to configure(MediaFormat, Surface, MediaCrypto, int) specifies the scaling mode to use.
final void signalEndOfInputStream()
Signals end-of-stream on input.
final void start()
After successfully configuring the component, call start.
final void stop()
Finish the decode/encode session, note that the codec instance remains active and ready to be start()ed again.
Protected Methods
void finalize()
Invoked when the garbage collector has detected that this instance is no longer reachable.
[Expand]
Inherited Methods
From class java.lang.Object

Constants

public static final int BUFFER_FLAG_CODEC_CONFIG

Added in API level 16

This indicated that the buffer marked as such contains codec initialization / codec specific data instead of media data.

Constant Value: 2 (0x00000002)

public static final int BUFFER_FLAG_END_OF_STREAM

Added in API level 16

This signals the end of stream, i.e. no buffers will be available after this, unless of course, flush() follows.

Constant Value: 4 (0x00000004)

public static final int BUFFER_FLAG_SYNC_FRAME

Added in API level 16

This indicates that the buffer marked as such contains the data for a sync frame.

Constant Value: 1 (0x00000001)

public static final int CONFIGURE_FLAG_ENCODE

Added in API level 16

If this codec is to be used as an encoder, pass this flag.

Constant Value: 1 (0x00000001)

public static final int CRYPTO_MODE_AES_CTR

Added in API level 16

Constant Value: 1 (0x00000001)

public static final int CRYPTO_MODE_UNENCRYPTED

Added in API level 16

Constant Value: 0 (0x00000000)

public static final int INFO_OUTPUT_BUFFERS_CHANGED

Added in API level 16

The output buffers have changed, the client must refer to the new set of output buffers returned by getOutputBuffers() from this point on.

Constant Value: -3 (0xfffffffd)

public static final int INFO_OUTPUT_FORMAT_CHANGED

Added in API level 16

The output format has changed, subsequent data will follow the new format. getOutputFormat() returns the new format.

Constant Value: -2 (0xfffffffe)

public static final int INFO_TRY_AGAIN_LATER

Added in API level 16

If a non-negative timeout had been specified in the call to dequeueOutputBuffer(MediaCodec.BufferInfo, long), indicates that the call timed out.

Constant Value: -1 (0xffffffff)

public static final String PARAMETER_KEY_REQUEST_SYNC_FRAME

Added in API level 19

Request that the encoder produce a sync frame "soon". Provide an Integer with the value 0.

Constant Value: "request-sync"

public static final String PARAMETER_KEY_SUSPEND

Added in API level 19

Temporarily suspend/resume encoding of input data. While suspended input data is effectively discarded instead of being fed into the encoder. This parameter really only makes sense to use with an encoder in "surface-input" mode, as the client code has no control over the input-side of the encoder in that case. The value is an Integer object containing the value 1 to suspend or the value 0 to resume.

Constant Value: "drop-input-frames"

public static final String PARAMETER_KEY_VIDEO_BITRATE

Added in API level 19

Change a video encoder's target bitrate on the fly. The value is an Integer object containing the new bitrate in bps.

Constant Value: "video-bitrate"

public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT

Added in API level 16

The content is scaled to the surface dimensions

Constant Value: 1 (0x00000001)

public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING

Added in API level 16

The content is scaled, maintaining its aspect ratio, the whole surface area is used, content may be cropped

Constant Value: 2 (0x00000002)

Public Methods

public void configure (MediaFormat format, Surface surface, MediaCrypto crypto, int flags)

Added in API level 16

Configures a component.

Parameters
format The format of the input data (decoder) or the desired format of the output data (encoder).
surface Specify a surface on which to render the output of this decoder.
crypto Specify a crypto object to facilitate secure decryption of the media data.
flags Specify CONFIGURE_FLAG_ENCODE to configure the component as an encoder.

public static MediaCodec createByCodecName (String name)

Added in API level 16

If you know the exact name of the component you want to instantiate use this method to instantiate it. Use with caution. Likely to be used with information obtained from MediaCodecList

Parameters
name The name of the codec to be instantiated.
Throws
IOException if the codec cannot be created.
IllegalArgumentException if name is null.

public static MediaCodec createDecoderByType (String type)

Added in API level 16

Instantiate a decoder supporting input data of the given mime type. The following is a partial list of defined mime types and their semantics:

  • "video/x-vnd.on2.vp8" - VP8 video (i.e. video in .webm)
  • "video/x-vnd.on2.vp9" - VP9 video (i.e. video in .webm)
  • "video/avc" - H.264/AVC video
  • "video/hevc" - H.265/HEVC video
  • "video/mp4v-es" - MPEG4 video
  • "video/3gpp" - H.263 video
  • "audio/3gpp" - AMR narrowband audio
  • "audio/amr-wb" - AMR wideband audio
  • "audio/mpeg" - MPEG1/2 audio layer III
  • "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged in LATM!)
  • "audio/vorbis" - vorbis audio
  • "audio/g711-alaw" - G.711 alaw audio
  • "audio/g711-mlaw" - G.711 ulaw audio

Parameters
type The mime type of the input data.
Throws
IOException if the codec cannot be created.
IllegalArgumentException if type is null.

public static MediaCodec createEncoderByType (String type)

Added in API level 16

Instantiate an encoder supporting output data of the given mime type.

Parameters
type The desired mime type of the output data.
Throws
IOException if the codec cannot be created.
IllegalArgumentException if type is null.

public final Surface createInputSurface ()

Added in API level 18

Requests a Surface to use as the input to an encoder, in place of input buffers. This may only be called after configure(MediaFormat, Surface, MediaCrypto, int) and before start().

The application is responsible for calling release() on the Surface when done.

The Surface must be rendered with a hardware-accelerated API, such as OpenGL ES. lockCanvas(android.graphics.Rect) may fail or produce unexpected results.

public final int dequeueInputBuffer (long timeoutUs)

Added in API level 16

Returns the index of an input buffer to be filled with valid data or -1 if no such buffer is currently available. This method will return immediately if timeoutUs == 0, wait indefinitely for the availability of an input buffer if timeoutUs < 0 or wait up to "timeoutUs" microseconds if timeoutUs > 0.

Parameters
timeoutUs The timeout in microseconds, a negative timeout indicates "infinite".

public final int dequeueOutputBuffer (MediaCodec.BufferInfo info, long timeoutUs)

Added in API level 16

Dequeue an output buffer, block at most "timeoutUs" microseconds. Returns the index of an output buffer that has been successfully decoded or one of the INFO_* constants below.

Parameters
info Will be filled with buffer meta data.
timeoutUs The timeout in microseconds, a negative timeout indicates "infinite".

public final void flush ()

Added in API level 16

Flush both input and output ports of the component, all indices previously returned in calls to dequeueInputBuffer(long) and dequeueOutputBuffer(MediaCodec.BufferInfo, long) become invalid.

public MediaCodecInfo getCodecInfo ()

Added in API level 18

Get the codec info. If the codec was created by createDecoderByType or createEncoderByType, what component is chosen is not known beforehand, and thus the caller does not have the MediaCodecInfo.

public ByteBuffer[] getInputBuffers ()

Added in API level 16

Call this after start() returns.

public final String getName ()

Added in API level 18

Get the component name. If the codec was created by createDecoderByType or createEncoderByType, what component is chosen is not known beforehand.

public ByteBuffer[] getOutputBuffers ()

Added in API level 16

Call this after start() returns and whenever dequeueOutputBuffer signals an output buffer change by returning INFO_OUTPUT_BUFFERS_CHANGED

public final MediaFormat getOutputFormat ()

Added in API level 16

Call this after dequeueOutputBuffer signals a format change by returning INFO_OUTPUT_FORMAT_CHANGED

public final void queueInputBuffer (int index, int offset, int size, long presentationTimeUs, int flags)

Added in API level 16

After filling a range of the input buffer at the specified index submit it to the component. Many decoders require the actual compressed data stream to be preceded by "codec specific data", i.e. setup data used to initialize the codec such as PPS/SPS in the case of AVC video or code tables in the case of vorbis audio. The class MediaExtractor provides codec specific data as part of the returned track format in entries named "csd-0", "csd-1" ... These buffers should be submitted using the flag BUFFER_FLAG_CODEC_CONFIG. To indicate that this is the final piece of input data (or rather that no more input data follows unless the decoder is subsequently flushed) specify the flag BUFFER_FLAG_END_OF_STREAM.

Parameters
index The index of a client-owned input buffer previously returned in a call to dequeueInputBuffer(long).
offset The byte offset into the input buffer at which the data starts.
size The number of bytes of valid input data.
presentationTimeUs The time at which this buffer should be rendered.
flags A bitmask of flags BUFFER_FLAG_SYNC_FRAME, BUFFER_FLAG_CODEC_CONFIG or BUFFER_FLAG_END_OF_STREAM.
Throws
MediaCodec.CryptoException if a crypto object has been specified in configure(MediaFormat, Surface, MediaCrypto, int)

public final void queueSecureInputBuffer (int index, int offset, MediaCodec.CryptoInfo info, long presentationTimeUs, int flags)

Added in API level 16

Similar to queueInputBuffer(int, int, int, long, int) but submits a buffer that is potentially encrypted.

Parameters
index The index of a client-owned input buffer previously returned in a call to dequeueInputBuffer(long).
offset The byte offset into the input buffer at which the data starts.
info Metadata required to facilitate decryption, the object can be reused immediately after this call returns.
presentationTimeUs The time at which this buffer should be rendered.
flags A bitmask of flags BUFFER_FLAG_SYNC_FRAME, BUFFER_FLAG_CODEC_CONFIG or BUFFER_FLAG_END_OF_STREAM.
Throws
MediaCodec.CryptoException if an error occurs while attempting to decrypt the buffer. An error code associated with the exception helps identify the reason for the failure.

public final void release ()

Added in API level 16

Make sure you call this when you're done to free up any opened component instance instead of relying on the garbage collector to do this for you at some point in the future.

public final void releaseOutputBuffer (int index, boolean render)

Added in API level 16

If you are done with a buffer, use this call to return the buffer to the codec. If you previously specified a surface when configuring this video decoder you can optionally render the buffer.

Parameters
index The index of a client-owned output buffer previously returned from a call to dequeueOutputBuffer(MediaCodec.BufferInfo, long).
render If a valid surface was specified when configuring the codec, passing true renders this output buffer to the surface.

public final void releaseOutputBuffer (int index, long renderTimestampNs)

If you are done with a buffer, use this call to update its surface timestamp and return it to the codec to render it on the output surface. If you have not specified an output surface when configuring this video codec, this call will simply return the buffer to the codec.

The timestamp may have special meaning depending on the destination surface.

SurfaceView specifics
If you render your buffer on a SurfaceView, you can use the timestamp to render the buffer at a specific time (at the VSYNC at or after the buffer timestamp). For this to work, the timestamp needs to be reasonably close to the current nanoTime(). Currently, this is set as within one (1) second. A few notes:
  • the buffer will not be returned to the codec until the timestamp has passed and the buffer is no longer used by the Surface.
  • buffers are processed sequentially, so you may block subsequent buffers to be displayed on the Surface. This is important if you want to react to user action, e.g. stop the video or seek.
  • if multiple buffers are sent to the Surface to be rendered at the same VSYNC, the last one will be shown, and the other ones will be dropped.
  • if the timestamp is not "reasonably close" to the current system time, the Surface will ignore the timestamp, and display the buffer at the earliest feasible time. In this mode it will not drop frames.
  • for best performance and quality, call this method when you are about two VSYNCs' time before the desired render time. For 60Hz displays, this is about 33 msec.

Parameters
index The index of a client-owned output buffer previously returned from a call to dequeueOutputBuffer(MediaCodec.BufferInfo, long).
renderTimestampNs The timestamp to associate with this buffer when it is sent to the Surface.

public final void setParameters (Bundle params)

Added in API level 19

Communicate additional parameter changes to the component instance.

public final void setVideoScalingMode (int mode)

Added in API level 16

If a surface has been specified in a previous call to configure(MediaFormat, Surface, MediaCrypto, int) specifies the scaling mode to use. The default is "scale to fit".

public final void signalEndOfInputStream ()

Added in API level 18

Signals end-of-stream on input. Equivalent to submitting an empty buffer with BUFFER_FLAG_END_OF_STREAM set. This may only be used with encoders receiving input from a Surface created by createInputSurface().

public final void start ()

Added in API level 16

After successfully configuring the component, call start. On return you can query the component for its input/output buffers.

public final void stop ()

Added in API level 16

Finish the decode/encode session, note that the codec instance remains active and ready to be start()ed again. To ensure that it is available to other client call release() and don't just rely on garbage collection to eventually do this for you.

Protected Methods

protected void finalize ()

Added in API level 16

Invoked when the garbage collector has detected that this instance is no longer reachable. The default implementation does nothing, but this method can be overridden to free resources.

Note that objects that override finalize are significantly more expensive than objects that don't. Finalizers may be run a long time after the object is no longer reachable, depending on memory pressure, so it's a bad idea to rely on them for cleanup. Note also that finalizers are run on a single VM-wide finalizer thread, so doing blocking work in a finalizer is a bad idea. A finalizer is usually only necessary for a class that has a native peer and needs to call a native method to destroy that peer. Even then, it's better to provide an explicit close method (and implement Closeable), and insist that callers manually dispose of instances. This works well for something like files, but less well for something like a BigInteger where typical calling code would have to deal with lots of temporaries. Unfortunately, code that creates lots of temporaries is the worst kind of code from the point of view of the single finalizer thread.

If you must use finalizers, consider at least providing your own ReferenceQueue and having your own thread process that queue.

Unlike constructors, finalizers are not automatically chained. You are responsible for calling super.finalize() yourself.

Uncaught exceptions thrown by finalizers are ignored and do not terminate the finalizer thread. See Effective Java Item 7, "Avoid finalizers" for more.