H265 SDP negotiation: level-id=180 from latest Chrome

[showier-drastic]

This is what I get from Chrome on macOS Version 144.0.7559.133 (Official Build) (arm64)

a=rtpmap:49 H265/90000r
a=rtcp-fb:49 goog-rembr
a=rtcp-fb:49 transport-ccr
a=rtcp-fb:49 ccm firr
a=rtcp-fb:49 nackr
a=rtcp-fb:49 nack plir
a=fmtp:49 level-id=180;profile-id=1;tier-flag=0;tx-mode=SRSTr
a=rtpmap:50 rtx/90000r
a=fmtp:50 apt=49r
a=rtpmap:51 H265/90000r
a=rtcp-fb:51 goog-rembr
a=rtcp-fb:51 transport-ccr
a=rtcp-fb:51 ccm firr
a=rtcp-fb:51 nackr
a=rtcp-fb:51 nack plir
a=fmtp:51 level-id=180;profile-id=2;tier-flag=0;tx-mode=SRSTr
a=rtpmap:52 rtx/90000r
a=fmtp:52 apt=51r

In the current code, H265 level is 156:

str0m/src/format/codec_config.rs

Line 276 in 5ff9b8f

self.add_h265(PT_H265, Some(PT_H265_RTX), 1, 0, 156); // Main, Main tier, Level 5.2

Therefore, negotiation fails. If I change the level-id to 180 manually in codec_config.rs, negotiation works.

[algesten]

Hi @showier-drastic! I haven't been deeply involved in the H265 stuff that's been happening. However I know @sr1990 has worked hard on these things. This PR which is very new, is working on the matching #853

It's still only in the main branch. Could you test against latest main to see if this is still an issue?

[showier-drastic]

@algesten Thanks for the quick reply! Actually I'm testing with the main branch, so even with this PR it doesn't work.

[showier-drastic]

Actually it sounds a little bit strange to me:

https://github.com/algesten/str0m/pull/853/changes#diff-3fb102dce4e1ebfbfcf2c341145d7bb2ae21ae9faed10376e74d6771d8872a57R438

    /// The level check `c1_level <= c0_level` enforces that the remote encoder
    /// must not exceed our decoder's configured level.

It seems that the code above assumes that the remote is the encoder and str0m is the decoder.

In my case, my str0m code is the encoder (which produces the video) and the browser (remote) is the decoder (which plays the video). So actually it seems that the limit should be reversed (or does not make sense here).

[algesten]

@showier-drastic interesting. So that comparison depends on direction. I wonder how that sits in relation to changing directions, documented here: https://github.com/algesten/str0m/blob/main/docs/SDP.md

[algesten]

@showier-drastic as a general rule-of-thumb the SDP is about "this is what I am prepared to receive", which always "wins" over the send side.

[showier-drastic]

To give some context, my application can be imagined as a network camera. I encode and send data on my app with str0m, and the browser receives the data. In my JavaScript design, it is the JS code that initiates a recvonly SDP offer, and str0m responds with a sendonly SDP answer. I found that with current main branch, even though Chrome send an offer with H265, level-id=180, str0m did not give a SDP answer with H265, because it is filtered out with match_h265_score.

Current code in str0m seems not considering the direction. It always puts str0m on c0 position, and remote on c1 position.

In src/format/codec_config.rs:

    pub fn match_params(&self, params: PayloadParams) -> Option<&PayloadParams> {
        let c = self.params.iter().max_by_key(|p| p.match_score(&params))?;
        c.match_score(&params)?; // avoid None, which isn't a match.
        Some(c)
    }

    pub(crate) fn sdp_match_remote(
        &self,
        remote_params: PayloadParams,
        remote_dir: Direction,
    ) -> Option<Pt> {
        // If we have no matching parameters locally, we can't accept the remote in any way.
        let our_params = self.match_params(remote_params)?;     // <========= remote_dir is not used here

        if remote_dir.sdp_is_receiving() {
            // The remote is talking about its own receive requirements. The PTs are not suggestions.
            Some(remote_params.pt())
        } else {
            // We can override the remote with our local config. We would have adjusted to
            // the remote earlier if we can.
            Some(our_params.pt())
        }
    }

And in src/format/payload_params.rs:

pub(crate) fn match_score(&self, o: &PayloadParams) -> Option<usize> {
        // we don't want to compare PT
        let c0 = self.spec;    // <============= self is always c0
        let c1 = o.spec;

        // ...
        if c0.codec == Codec::H265 {
            return Self::match_h265_score(c0, c1);
        }

    }

[algesten]

I looked into this together with the recent PR #853 changes and the SDP negotiation model in str0m (receiver wins). Here's what I think is going on:

The level check in match_h265_score should not reject on level mismatch

The current code at payload_params.rs:475 rejects when the remote level exceeds the local level:

if c1_ptl.level() > c0_ptl.level() {
    return None;
}

This is wrong for both SDP direction cases:

Case 1: sendonly/recvonly

The receiver's level is authoritative ("receiver wins"). The sender's level is just a preference — the sender adapts to the receiver's stated capability. A level mismatch is not a compatibility problem.

For example: str0m recvonly at Level 5.2, remote sendonly at Level 6.0 — str0m's Level 5.2 wins, the remote must send at <=5.2. Match should succeed.

Case 2: sendrecv/sendrecv

Both sides are stating their receive capabilities. Both will also send. The effective level must narrow to the LCD (lowest common denominator): min(local_level, remote_level).

For example: str0m at Level 5.2, Chrome at Level 6.0 — effective session level is 5.2. Still not incompatible — it's a negotiation.

In both cases, profile + tier match is sufficient for codec compatibility

Level differences should not gate the match. They may influence scoring (prefer closer levels), but a return None on level mismatch is incorrect.

Second issue: update_param blindly copies the remote's PTL

In payload_params.rs:541-542, the negotiation step copies the remote's full PTL:

if let Some(ptl) = first.spec.format.h265_profile_tier_level {
    self.spec.format.h265_profile_tier_level = Some(ptl);

For case 2 (sendrecv/sendrecv), this should narrow to the LCD — take min(local_level, remote_level) while keeping the matched profile/tier — so the SDP answer reflects the negotiated level, not the remote's higher capability.

Summary of needed fixes

1. match_h265_score: Remove the c1_level > c0_level -> None rejection. Profile + tier match should be sufficient. Keep level difference as a score penalty for preferring closer matches.
2. update_param: For H.265, narrow the level to min(local, remote) instead of blindly copying the remote's PTL.
3. The default level (currently 156 / Level 5.2 in codec_config.rs:276) could also be bumped to 180 / Level 6.0 to better match what modern browsers advertise, but this alone wouldn't fix the underlying logic issues.

[showier-drastic]

Instead of Keep level difference as a score penalty for preferring closer matches, is it better to make the matching code direction aware?

[showier-drastic]

Also, instead of bumped to 180 / Level 6.0, the level might be better supplied by the user (because the user knows their encoder / decoder capability) instead of being hard-coded in the code.

[algesten]

Instead of Keep level difference as a score penalty for preferring closer matches, is it better to make the matching process direction aware?

I'm not sure :)

Also, instead of bumped to 180 / Level 6.0, the level might be better supplied by the user (because the user knows their encoder / decoder capability) instead of being hard-coded in the code.

Yeah, the enable functions in the config could take this as an argument.

[algesten]

Instead of Keep level difference as a score penalty for preferring closer matches, is it better to make the matching code direction aware?

My main headache is about direction changes. In some respects it's like we always need to treat it as if it's sendrecv because it can change.

[sr1990]

Hey guys, thanks for catching these. I’m still getting up to speed on the project and RTC specs. I had tested the changes locally on Chrome on macOS.