Add detection for DTS-HD audio tracks in MatroskaExtractor

[RyuzakiKK]

Distinguish between DTS and DTS-HD formats by analyzing the frame data in matroska containers. This enables accurate MIME type assignment for DTS-HD contents.

This fixes the old issue google/ExoPlayer#6225.

The patch is based on the initial work of @moneytoo.

[tonihei]

Thanks for the contribution!

Do you happen to have sample files that show the parsing issue to verify the fix works? You can share them to [email protected] with "Issue #2485" in the subject and report back here.

For the actual implementation, I was wondering how this new path relates to the existing MatroskaExtractor.Track.initializeOutput method that usually builds the format. Is the new change building an additional track after the regular DTS track has already been built?

[RyuzakiKK]

@tonihei I just sent an email with two sample files, one for DTS:X and one for DTS-HD MA.

In both cases Media3 will reproduce them as DTS, while with this PR it should correctly reproduce them as DTS:X and DTS-HD MA respectively.

As far as I know, with the Matroska container the only way to distinguish between the different DTS formats is by analyzing the frame data. This is why I don't think it was feasible to add this check to MatroskaExtractor.Track.initializeOutput.

[tonihei]

None of the two files seem to trigger the new track.output.format call as far as I see? Could you check again that it works for you? When reaching this point, the source is also already "prepared", so we wouldn't be able to add new formats to the list (this is what I wanted highlight in my first comment). So even if the format call is triggered, this probably still needs to some refactor to make it wait with creating tracks until all the necessary information is known.

[RyuzakiKK]

None of the two files seem to trigger the new track.output.format call as far as I see? Could you check again that it works for you?

Oops sorry. When I did a clean up before opening the PR I introduced a bug and forgot to do a final test. With the latest revision it should work fine.

I tested it with the two samples that I sent you via email and the demo media3 app. In both cases you can see that the MKV gets loaded with audio/vnd.dts (in the top left corner log) and after a few ms it changes to the correct audio/vnd.dts.hd.
And my soundbar correctly signals that the audio is coming through respectively as DTS:X and DTS-HD.

When reaching this point, the source is also already "prepared", so we wouldn't be able to add new formats to the list (this is what I wanted highlight in my first comment). So even if the format call is triggered, this probably still needs to some refactor to make it wait with creating tracks until all the necessary information is known.

For me it seems to be always working fine, but it could be a fluke with the timings.

Do you have suggestions on exactly where this new check should be moved into? I can try to improve it, but of course if you prefer to take this on it would be great.

[tonihei]

With the latest revision it should work fine.

Thanks, I can see the TrackOutput.format path being triggered now as well.

For me it seems to be always working fine, but it could be a fluke with the timings.

It's not a fluke I'm afraid. What is happening at the moment is that the player is prepared with a regular DTS track first (you can see the Track:0, id=2, mimeType=audio/vnd.dts output for example in the EventLogger tracks log). The player then performs track selection and output path configuration based on this format. Only slightly later do we get an updated format from the new code, but this only updates the internal SampleStream and the already existing output path without going through any track selection steps or state updates. You can tell by not seeing the new format in the EventLogger tracks, but only in downstreamFormat logs.

The proper fix would be to not call extractorOutput.endTracks from MatroskaExtractor until we fully determined the track's formats. (and maybe don't call extractorOutput.format either, but this should actually be fine as long as we haven't extracted any real samples yet). Maybe you could set a boolean flag waitingForDtsAnalysis in the Track class instead and not send the endTracks call until this is done?

[RyuzakiKK]

The proper fix would be to not call extractorOutput.endTracks from MatroskaExtractor until we fully determined the track's formats. (and maybe don't call extractorOutput.format either, but this should actually be fine as long as we haven't extracted any real samples yet). Maybe you could set a boolean flag waitingForDtsAnalysis in the Track class instead and not send the endTracks call until this is done?

Right, makes sense. Thanks for the review.

@tonihei I implemented it in the latest revision of this PR.
I included a small maybeEndTracks() function to check if we are ready to call endTracks.

(I left the extractorOutput.format() call as-is. Please let me know if you preferred to change that as well)

[tonihei]

Thanks for the update! I'll start the internal merge process now.

[tonihei]

Do you have any test samples we could use in our extractor tests? I tried to convert the existing sample_dts_hd.ts with ffmpeg, but it doesn't seem to contain valid samples with the FRAME_TYPE_CORE after the conversion (you can see the tests I added produce the wrong mime type still).

[RyuzakiKK]

Do you have any test samples we could use in our extractor tests? I tried to convert the existing sample_dts_hd.ts with ffmpeg, but it doesn't seem to contain valid samples with the FRAME_TYPE_CORE after the conversion (you can see the tests I added produce the wrong mime type still).

@tonihei these are two samples that should have valid DTS-HD MA and DTS-HD MA + DTS:X
mediainfo is correctly reporting on both of them, but I didn't test them yet with media3.

dts_samples.zip

[tonihei]

Thanks, these files work!