Gapless playback is not working

[adoudech]

Issue description

When trying to play a gapless playback between two MP3 files, a noticeable gap is detected between them.
The issue is also reproduced when using Exoplayer demo app with the test.mp3 file already added in assets.
I added the following snippet to PlayerActivity in Exoplayer demo app L299 for test purpose => the issue is reproduced : a gap between the two media sources is heard.

    val concatenatingMediaSource = ConcatenatingMediaSource()
    val defaultMediaSourceFactory = DefaultMediaSourceFactory(context)
    concatenatingMediaSource.addMediaSource(defaultMediaSourceFactory.createMediaSource(
        MediaItem.fromUri("asset:///test.mp3")))
    concatenatingMediaSource.addMediaSource(defaultMediaSourceFactory.createMediaSource(
        MediaItem.fromUri("asset:///test.mp3")))
    player.setMediaSource(concatenatingMediaSource)
    player.prepare()

Please notice that the delay/padding information is well detected by Exoplayer for the test.mp3 in demo app assets.

• ExoPlayer version number : 2.13.0
• Android version : Android 7 & Android 10
• Android device: Samsung galaxy s7 edge & Samsung galaxy s10

Please let me know if you need more details.
Thank you

[christosts]

@krocard can you take a look?

[krocard]

I can't reproduce on a Pixel 3 on Android 10. Here are my exact steps:

git clone -b r2.13.0 https://github.com/google/ExoPlayer.git && cd ExoPlayer
ANDROID_SDK_ROOT=~/Android/Sdk/ ./gradlew installNoDecoderExtensionsDebug
adb push testdata/src/test/assets/media/mp3/test.mp3 /sdcard/
adb shell am start -a com.google.android.exoplayer.demo.action.VIEW_LIST --es uri_0 file:///sdcard/test.mp3 --es uri_1 file:///sdcard/test.mp3

The file plays twice without any audible gap. I also tried to apply a similar change to yours (though test.mp3 is not in the demo asset, so I used a file URI):

-    player.setMediaItems(mediaItems, /* resetPosition= */ !haveStartPosition);
+    ConcatenatingMediaSource concatenatingMediaSource = new ConcatenatingMediaSource();
+    DefaultMediaSourceFactory defaultMediaSourceFactory = new DefaultMediaSourceFactory(this);
+    concatenatingMediaSource.addMediaSource(defaultMediaSourceFactory.createMediaSource(
+        MediaItem.fromUri("file:///sdcard/test.mp3")));
+    concatenatingMediaSource.addMediaSource(defaultMediaSourceFactory.createMediaSource(
+        MediaItem.fromUri("file:///sdcard/test.mp3")));
+    player.setMediaSource(concatenatingMediaSource);

Please follow those steps (preferably the first ones a they do not involve any code change) and attach a bug report if you still observe the issue.

[adoudech]

I'm still facing the issue after following your steps (the first ones) with my device Samsung galaxy s10.
I tested on emulator Pixel 3 Android 10, I still got the gap between the two test mp3 files.

[adoudech]

Please take a look at the attached file :

Screen_Recording_20210216-185257_ExoPlayer.mp4

[krocard]

I was able to reproduce on the emulator (pixel 3 Android 10). I have not yet traced the issue but I can hear underrun though that seem unrelated.
Longer gapless media do not exhibit the problem, so the issue could be cause/revealed by the short duration of this test media.

[GeoffreyMetais]

I confirm this behavior with test.mp3 and Exoplayer r2.13.0.

I clearly hear a gap with a Samsung Galaxy S8 (Android 9)

But no gap at all on Pixel 4a and galaxy S20+ (both Android 11)

[adoudech]

I upgraded my Samsung galaxy S10 to Android 11 and I cannot reproduce the issue => Gapless playback seems working on Android 11.
/!\ I still have the issue on Samsung galaxy s7 Android 7

[adoudech]

@krocard Any updates on this issue please? Thanks for any suggestions/clues

[krocard]

I found what is happening but I have no root caused it yet.

Normally (for example on Pixel) during a track transition here is what happens:

1. Extractor reaches the end of the 1st track. Last MP3 packet is at Presentation Time τ
2. Renderer is notified that the current stream will end at τ.
3. Renderer pulls from the MP3 buffer queue the last buffer of the 1st track (It has the Presentation Time τ). It queues it for decoding.
4. Renderer pulls from the MP3 buffer queue a format change (due to the switch to the second track). It stops pulling from the MP3 buffer queue until τ is played and saves the new input format.
5. Renderer pulls from the decoder a buffer τ and plays it. From step 2) it knows that τ was the last buffer of the stream. It notifies the AudioSink that there is now an audio discontinuity.
6. Renderer pulls from the MP3 buffer queue the first buffer of the second track, which has a Presentation Time τ'.
7. Renderer pushes the τ' buffer for decoding.
8. Renderer pulls from the decoder a buffer τ'. From step 6, because of the buffer presentation time, the renderer knows that it is the first buffer of a new format. It reconfigure the AudioSink with the new format and pushes the τ' buffer to it.
9. AudioSink receives the τ' buffer. It knows there has been a reconfiguration is pending, so drains its AudioProcessors before reconfiguring.
10. AudioSink's TrimmingProcessor knows that a reconfiguration is pending from 8. It concludes that the previous track has ended. Therefore it drops the amount of data required by the gapless metadata.
11. AudioSink the handle the buffer, but as there is a discontinuity pending from step 5, it drains the AudioProcessors again*. This is a no-op, but confusing.

The issue I'm observing in the emulator is at step 5. The renderer assumes that if the last buffer of the track queued for decoding has a presentation time of τ, the last buffer decoded of the codec will have a presentation time of τ. This is not the case on the emulator. An additional decoded buffer with a presentation time τ'' is outputted by the codec even though no encoded buffer with a presentation time τ'' was ever pushed.
Note that presentation time τ'' == τ'. I don't think it is significant tough.

Here is what happens on the emulator:

1. same
2. same
3. same
4. same
5. same, because Renderer assumes (incorrectly) that τ is the last buffer of track 1 from step 2, it informs the audio sink that there is a discontinuity. This is incorrect and leads to the following
   1. Render pulls the extra τ'' buffer (which is the real last buffer of track 1) from the decoder. It sends it to AudioSink for playing.
   2. AudioSink receives the τ'' buffer. It was told in step 5 that there is a discontinuity, so it drains its internal processors.
   3. AudioSink's TrimmingProcessor has no reconfiguration pending. It concludes that this is a seek not the end of the track. Therefore it does not drop the amount of data required by the gapless metadata and plays it instead. => a gap is heard

Following reproduction step from #8594 (comment). The consistent presentation times observed are:

• τ=1018775 us
• τ'=τ''=1044897us

I don't know why the decoder (OMX.google.mp3.decoder) outputs one more buffer than was queued. We might want to rely on the EOS flag on the buffer rather than the presentation time.

Regardless, we should avoid the constant double onQueueEndOfStream call on the processors on each track transition in handle buffer. One due to reconfigurationPending (10) and the second one due to the startMediaTimeUsNeedsSync (11) due to handleDiscontinuity called on track transition (5).

[krocard]

Log of the Emulator at the track transition (gap is heard)

MRend.drainOutputBuffer: new outputBuffer pt=835918
MRend.feedInputBuffer: Queuing input buffer ts=1018775
MRend.drainOutputBuffer: new outputBuffer pt=862040
MRend.onStreamChanged: Queueing stream changed for pt=1018775
MRend.feedInputBuffer: sample stream format change
MRend.onInputFormatChanged: waitingForFirstSampleInFormat = true
MRend.drainOutputBuffer: new outputBuffer pt=888163
MRend.drainOutputBuffer: new outputBuffer pt=914285
MRend.drainOutputBuffer: new outputBuffer pt=940408
MRend.drainOutputBuffer: new outputBuffer pt=966530
MRend.drainOutputBuffer: new outputBuffer pt=992653
MRend.drainOutputBuffer: new outputBuffer pt=1018775
MRend.onProcessedOutputBuffer: calling onProcessedStreamChange pt=1018775 pendingOutputStreamOffsetCount=1 presentationTimeUs=1018775 pendingOutputStreamSwitchTimesUs[0]=1018775
ASink.handleDiscontinuity: startMediaTimeUsNeedsSync = true
MRend.drainOutputBuffer: new outputBuffer pt=1044897
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=true
TrimP.onQueueEndOfStream: end of stream queued but no reconfiguration!
ASink.drainToEndOfStream: return false as AP is not ended
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=false
ASink.drainToEndOfStream: return false as AP is not ended
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=false
ASink.drainToEndOfStream: return false as AP is not ended
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=false
TrimP.getOutput: outputted the endBuffer, NO trimming!
ASink.drainToEndOfStream: return false as still buffer to write
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: return false as still buffer to write
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: return false as still buffer to write
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: return false as still buffer to write
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: completed
TrimP.onFlush: without reconfiguration pending
MRend.feedInputBuffer: Adding new format to queue for ts=1044897 due to waitingForFirstSampleInFormat
MRend.feedInputBuffer: Queuing input buffer ts=1044897
MRend.feedInputBuffer: Queuing input buffer ts=1071019
MRend.feedInputBuffer: Queuing input buffer ts=1097141
MRend.feedInputBuffer: Queuing input buffer ts=1123264
MRend.feedInputBuffer: Queuing input buffer ts=1149386
MRend.feedInputBuffer: Queuing input buffer ts=1175509
MRend.feedInputBuffer: Queuing input buffer ts=1201631
MRend.feedInputBuffer: Queuing input buffer ts=1227754
MRend.drainOutputBuffer: new outputBuffer pt=1044897
MRend.updateOutputFormatForTime: Output format changed from format queue pt=1044897 Format(null, null, null, audio/mpeg, null, -1, null, [-1, -1, -1.0], [1, 44100])
ARend.onOutputFormatChanged: reconfigure sink due to output format change
TrimP.onConfigure: trim=576, 1404
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=true
TrimP.onQueueEndOfStream: Trimming 529 trimmedFrameCount=1105
ASink.drainToEndOfStream: completed
TrimP.onFlush: will now skip 576 at beginning and 1404 at end
TrimP.queueInput: All start bytes have been trimmed
MRend.drainOutputBuffer: new outputBuffer pt=1071019
MRend.drainOutputBuffer: new outputBuffer pt=1097141
MRend.drainOutputBuffer: new outputBuffer pt=1123264
MRend.drainOutputBuffer: new outputBuffer pt=1149386
MRend.drainOutputBuffer: new outputBuffer pt=1175509
MRend.drainOutputBuffer: new outputBuffer pt=1201631
MRend.feedInputBuffer: Queuing input buffer ts=1253876
MRend.feedInputBuffer: Queuing input buffer ts=1279999
MRend.feedInputBuffer: Queuing input buffer ts=1306121

Log of the Pixel 3 (gapless)

MRend.feedInputBuffer: Queuing input buffer ts=980657
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1018775 to=1006780
MRend.feedInputBuffer: Queuing input buffer ts=1006780
MRend.drainOutputBuffer: new outputBuffer pt=876167
MRend.drainOutputBuffer: new outputBuffer pt=902290
MRend.drainOutputBuffer: new outputBuffer pt=928412
MRend.drainOutputBuffer: new outputBuffer pt=954535
MRend.drainOutputBuffer: new outputBuffer pt=980657
MRend.onStreamChanged: Queueing stream changed for pt=1018775
MRend.feedInputBuffer: sample stream format change
MRend.onInputFormatChanged: waitingForFirstSampleInFormat = true
MRend.drainOutputBuffer: new outputBuffer pt=1006780
MRend.drainOutputBuffer: new outputBuffer pt=1032902
MRend.onProcessedOutputBuffer: calling onProcessedStreamChange pt=1032902 pendingOutputStreamOffsetCount=1 presentationTimeUs=1032902 pendingOutputStreamSwitchTimesUs[0]=1018775
ASink.handleDiscontinuity: startMediaTimeUsNeedsSync = true
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1044897 to=1044897
MRend.feedInputBuffer: Adding new format to queue for ts=1044897 due to waitingForFirstSampleInFormat
MRend.feedInputBuffer: Queuing input buffer ts=1044897
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1071019 to=1059023
MRend.feedInputBuffer: Queuing input buffer ts=1059023
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1097141 to=1085146
MRend.feedInputBuffer: Queuing input buffer ts=1085146
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1123264 to=1111268
MRend.feedInputBuffer: Queuing input buffer ts=1111268
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1149386 to=1137391
MRend.feedInputBuffer: Queuing input buffer ts=1137391
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1175509 to=1163513
MRend.feedInputBuffer: Queuing input buffer ts=1163513
MRend.feedInputBuffer: c2Mp3TimestampTracker changed ts from=1201631 to=1189636
MRend.feedInputBuffer: Queuing input buffer ts=1189636
MRend.drainOutputBuffer: new outputBuffer pt=1044897
MRend.updateOutputFormatForTime: Output format changed from format queue pt=1044897 Format(null, null, null, audio/mpeg, null, -1, null, [-1, -1, -1.0], [1, 44100])
ARend.onOutputFormatChanged: reconfigure sink due to output format change
TrimP.onConfigure: trim=576, 1404
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=true
TrimP.onQueueEndOfStream: Trimming 1404 trimmedFrameCount=1980
ASink.drainToEndOfStream: completed
TrimP.onFlush: will now skip 576 at begining and 1404 at end
ASink.handleBuffer: calling drainToEndOfStream() due to startMediaTimeUsNeedsSync
ASink.drainToEndOfStream: drainingAudioProcessorIndex=0 audioProcessorNeedsEndOfStream=true
TrimP.onQueueEndOfStream: end of stream queued but no reconfiguration!
ASink.drainToEndOfStream: completed
TrimP.onFlush: without reconfiguration pending
TrimP.queueInput: All start bytes have been trimmed
MRend.drainOutputBuffer: new outputBuffer pt=1059023
MRend.drainOutputBuffer: new outputBuffer pt=1085146
MRend.drainOutputBuffer: new outputBuffer pt=1111268

[internal: patch adding logging at cl/359504935]