Unsupported media is not detected and MediaCodec renderer raises exception

[mrqinshou]

-- Edited by christosts --

When trying to play this MP4 file
http://www.mrqinshou.com:7000/file/img/tmp.mp4

ExoPlayer detects that it can be played by the MediaCodecRenderer, event logger output

MediaCodecVideoRenderer [
        Group:0, adaptive_supported=N/A [
          [X] Track:0, id=1, mimeType=video/avc, res=480x480, fps=25.0, supported=YES
        ]
      ]

but later the MediaCodec raises an excpetion, with the following stacktrace

some title

``` 2020-04-28 10:13:38.564 21122-21177/com.qinshou.qinshoubox I/MediaCodec: [OMX.qcom.video.decoder.avc] setting surface generation to 21628929 2020-04-28 10:13:38.565 21122-21178/com.qinshou.qinshoubox I/ExtendedACodec: setupVideoDecoder() 2020-04-28 10:13:38.566 21122-21178/com.qinshou.qinshoubox I/ExtendedACodec: Decoder will be in frame by frame mode 2020-04-28 10:13:38.566 21122-21178/com.qinshou.qinshoubox W/ExtendedACodec: Failed to set frame packing format on component 2020-04-28 10:13:38.584 21122-21183/com.qinshou.qinshoubox I/OMXClient: Treble IOmx obtained 2020-04-28 10:13:38.585 21122-21183/com.qinshou.qinshoubox I/ACodec: codec does not support config priority (err -2147483648) 2020-04-28 10:13:38.585 21122-21183/com.qinshou.qinshoubox I/ACodec: codec does not support config priority (err -2147483648) 2020-04-28 10:13:38.586 21122-21183/com.qinshou.qinshoubox W/ExtendedACodec: Failed to get extension for extradata parameter 2020-04-28 10:13:38.586 21122-21178/com.qinshou.qinshoubox E/ACodec: [OMX.qcom.video.decoder.avc] ERROR(0x80001009) 2020-04-28 10:13:38.586 21122-21178/com.qinshou.qinshoubox E/ACodec: signalError(omxError 0x80001009, internalError -2147483648) 2020-04-28 10:13:38.586 21122-21177/com.qinshou.qinshoubox E/MediaCodec: Codec reported err 0x80001009, actionCode 0, while in state 6 2020-04-28 10:13:38.595 21122-21174/com.qinshou.qinshoubox E/ExoPlayerImplInternal: Renderer error: index=0, type=video, format=Format(1, null, null, video/avc, null, -1, null, [480, 480, 25.0], [-1, -1]), rendererSupport=YES com.google.android.exoplayer2.ExoPlaybackException: java.lang.IllegalStateException at com.google.android.exoplayer2.BaseRenderer.createRendererException(BaseRenderer.java:359) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:732) at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:599) at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:329) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:164) at android.os.HandlerThread.run(HandlerThread.java:65) Caused by: java.lang.IllegalStateException at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method) at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:2639) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:1515) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:718) at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:599)  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:329)  at android.os.Handler.dispatchMessage(Handler.java:102)  at android.os.Looper.loop(Looper.java:164)  at android.os.HandlerThread.run(HandlerThread.java:65)  2020-04-28 10:13:38.596 21122-21174/com.qinshou.qinshoubox E/ExoPlayerImplInternal: Disable failed. java.lang.IllegalStateException at android.media.MediaCodec.native_flush(Native Method) at android.media.MediaCodec.flush(MediaCodec.java:2121) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.flushOrReleaseCodec(MediaCodecRenderer.java:773) at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.flushOrReleaseCodec(MediaCodecVideoRenderer.java:724) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onDisabled(MediaCodecRenderer.java:634) at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onDisabled(MediaCodecVideoRenderer.java:569) at com.google.android.exoplayer2.BaseRenderer.disable(BaseRenderer.java:168) at com.google.android.exoplayer2.ExoPlayerImplInternal.disableRenderer(ExoPlayerImplInternal.java:1136) at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:891) at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:850) at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:382) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:164) at android.os.HandlerThread.run(HandlerThread.java:65) 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox E/Surface: getSlotFromBufferLocked: unknown buffer: 0x75c46a8a80 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox W/ACodec: [OMX.qcom.video.decoder.avc] can not return buffer 20 to native window 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox E/Surface: getSlotFromBufferLocked: unknown buffer: 0x75c46a8b00 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox W/ACodec: [OMX.qcom.video.decoder.avc] can not return buffer 19 to native window 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox E/Surface: getSlotFromBufferLocked: unknown buffer: 0x75c46a8b80 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox W/ACodec: [OMX.qcom.video.decoder.avc] can not return buffer 18 to native window 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox E/Surface: getSlotFromBufferLocked: unknown buffer: 0x75c46a8c00 2020-04-28 10:13:38.600 21122-21178/com.qinshou.qinshoubox W/ACodec: [OMX.qcom.video.decoder.avc] can not return buffer 17 to native window 2020-04-28 10:13:38.606 21122-21174/com.qinshou.qinshoubox E/ExoPlayerImplInternal: Reset failed. java.lang.IllegalStateException at android.media.MediaCodec.native_stop(Native Method) at android.media.MediaCodec.stop(MediaCodec.java:2074) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.releaseCodec(MediaCodecRenderer.java:668) at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.releaseCodec(MediaCodecVideoRenderer.java:714) at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onReset(MediaCodecRenderer.java:641) at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onReset(MediaCodecVideoRenderer.java:578) at com.google.android.exoplayer2.BaseRenderer.reset(BaseRenderer.java:175) at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:900) at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:850) at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:382) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:164) at android.os.HandlerThread.run(HandlerThread.java:65) ```

The video is not playable by other native apps in Android, but vlc and mxplayer play it with external decoders. Inspecting vlc logs, it is clearly detected that MediaCodec does not support this file and switches to ffmpeg decoder.

We should fix this part, the fact that ExoPlayer does not identify this format is not supported. Debugging this, when the renderer is queried whether the Format is supported, Format.codecs is null; we can start by looking at why this occurs.

[christosts]

The provided mp4 file is encoded using the AVC High 4:4:4 Predictive. It is not guaranteed that such content will be playable using the platform's decoders, as the Main Profile is the only profiled required to work on Android
https://developer.android.com/guide/topics/media/media-formats#video-codecs

Please try to re-encode the content using the Main Profile.

[christosts]

I do see on the exoplayer event logger however that it is not clearly detected that the media is unsupported or exceeds the device capabilities. I am keeping this issue open to track a fix on this.

[christosts]

There is already an issue tracking the same fix, #5882, so I will close this one in favor of #5882.

@mrqinshou: to sum up, the root of the problem is that the particular file cannot be played using the Android platform decoders. You need to re-encode the file in a format supported by Android. Alternatively, there is an open pull request to enable video decoding with ffmpeg in ExoPlayer which is currently on its way to the dev-v2 branch. (#7132) You can try to either build with the pull request or wait until it's pushed to dev-v2 and try to play the content with the ffmpeg decoder.