mvhd.scale and mdhd.timescale not equal and edit list

[RufaelDev]

Some of the files

https://dash.akamaized.net/WAVE/vectors/releases/1/caaa_sets/he_aac/at9/2023-04-27/stream.mpd
https://dash.akamaized.net/WAVE/vectors/releases/1/camc_sets/aac_lc/at11/2023-04-27/stream.mpd

etc have different values, but the edit list is present. I have more info on request.

Strictly speaking this is not a validation but edit according to 14496-12 is on movie timescale, so the edit is not on the mdhd timeline, this may break some of the audio priming even though we have seen this in other implementations.

Easiest way to solve is to make mdhd.timescale and mvhd.timescale should be equal.

[rbouqueau]

I confirm the values are outliers (>5s delay on the audio track!).

Who has access to the streams before being DASHed? I guess the error comes from the source but I can't check that.

I don't even know how these were generated. Could it documented somehow in the README?

[rbouqueau]

cf #58

[jpiesing]

@yanj-github @rcottingham ?

[rbouqueau]

@yanj-github @rcottingham No news on this? Does cta-wave/Test-Content#51 fix this issue?

[michael-forsyth]

The edit list box is interesting in that it uses two timescales

from iso 14496-12 2015 version

section 8.6.6.3
"segment_duration is an integer that specifies the duration of this edit segment in units of the
timescale in the Movie Header Box"
"media_time is an integer containing the starting time within the media of this edit segment (in
media time scale units, in composition time). If this field is set to –1, it is an empty edit. The last
edit in a track shall never be an empty edit. Any difference between the duration in the Movie
Header Box, and the track’s duration is expressed as an implicit empty edit at the end."

The media timescale is defined in the 'mdhd' box not the 'mvhd' box
section 8.4.2.3
"timescale is an integer that specifies the timescale for this media; this is the number of time
units that pass in one second. For example, a time coordinate system that measures time in
sixtieths of a second has a time scale of 60."

In summary:
segment_duration uses the timescale from the 'mvhd' box
media_time uses the timescale from the 'mdhd' box
It is reasonable normal for these timescales to be different

[RufaelDev]

@michael-forsyth I follow your interpretation, our reading was following that all fields follow movie timeline/timescale as there is no explicit reference to 8.4.3 or track timeline. Also media timeline can refer to any timeline not only media track. I agree that many implementations use the mdhd timescale which is good. However clause 8.4.2.3 does not explicitly define media timeline, the definition is used throughout the document for different timelines.

I just wanted to clarify this point, maybe as there is a new version of 14496-12 under ballot it is possible to make a comment and make it more explicit, and leave the test content as is. There may be some incorrect implementations that interpret the edit as a 2 second audio edit. In this case we can leave the test files as is.

[rbouqueau]

I agree with @michael-forsyth interpretation.

Seems like GPAC does this too:

$ gpac -i https://dash.akamaized.net/WAVE/vectors/releases/1/caaa_sets/he_aac/at9/2023-04-27/stream.mpd#audio inspect
PID 1 audio inMovie duration 00:30.000 timescale 48000 delay -5058 48000 Hz 2 channels 128 kbps codec mp4a.40.2 AAC LC (aot=2 backward compatible) HEAACv1 SBR 48000 Hz aot MPEG-4 Audio SBR

=> delay -5058ms

Is it the expected value?

[jpiesing]

Please can someone open an MPEG git issue on this ambiguity?

[rbouqueau]

@jpiesing I don't think there is any normative issue. It would have been there for 20 years which is unlikely for a deployed feature. That being said clarifications are always welcome.

Edit lists are not easy to grab. They are complex because they are made to deal with complex non-linear scenarios. That's why there are many errors done. And that's why some specs try to avoid using (or restricting) them.

In this issue, IIUC:

• @RufaelDev reports that it would be desirable to make both timescales equal to mitigate errors.
• I report that with the spec interpretation the current content shows a delay of -5058ms. I'm asking for confirmation from Resilion that this is the expected value.

[RufaelDev]

@rbouqueau it seems that gpac uses mvhd timescale instead of mdhd timescale as pointed out by @michael-forsyth, or is the gpac output based on the timescale and not milli seconds ?

I think clarifying the point made by @michael-forsyth in MPEG file format makes sense (i.e. no normative change just pointing this out explicitly),

then for CMAF I think a question should be if it makes sense to adopt the rule that mvhd.timescale == mdhd.timescale as CMAF is single track so it is not clear why these two values should be different.

And yes I will attend the next MPEG meeting and review the 14496-12 version under ballot!

[rbouqueau]

The spec says:

• edit_duration -> movie header timescale
• media_time -> media header timescale

GPAC does that.

[RufaelDev]

spec does not say that explicitly and why then media time 5058 gives you 5058 ms instead of 5058/48000 delay in gpac ?

[rbouqueau]

Ahhh you are right. Sorry for disturbing your issue!! Let's focus on your initial message then.

[jpiesing]

This issue has been floating around for over 4 months.
Can we just close it? If not, who should comment or do something towards someone else?