concern regarding TLS extension sorting

I came across this today:

The newer one, "tls/1", sorts the extensions into lexicographic order, to compensate for the randomization of those fields introduced by some TLS clients.

https://github.com/cisco/mercury/blob/main/doc/npf.md#tls

which is concerning. some servers, such as android.googleapis.com, are sensitive to the extension order. This is not theoretical. for example, using these extensions:

&RenegotiationInfoExtension{},
&SNIExtension{},
&UtlsExtendedMasterSecretExtension{},
&SessionTicketExtension{},
&SignatureAlgorithmsExtension{},
&StatusRequestExtension{},
&ALPNExtension{},
&SupportedPointsExtension{},
&SupportedCurvesExtension{},

I get this result:

200 OK

if I swap the first two, I get this:

403 Forbidden

So if extensions are sorted, then an NPF cannot be used to reconstruct the original client hello, which would make it useless for actually making network requests. So imagine this conversation:

A: hello I cannot connect to this server, can anyone help?
B: sure just use tls/1...
A: it doesn't work...
B: ...

Also, this is confusing:

The "tls/1" fingerprint format is

   "tls/1" (TLS_Version) (TLS_Ciphersuite) [ QUIC_Extension* ]

and the older "tls" fingerprint format is

   "tls/" (TLS_Version) (TLS_Ciphersuite) ((TLS_Extension)*)

supposedly tls/1 sorts the extensions, but it also lists QUIC_Extension. does that mean only QUIC hellos can be sorted?

[davidmcgrew]

Sort-normalization is used for both QUIC and TLS when the "tls/1" fingerprint format is in use. QUIC_Extension is included in the TLS fingerprint specification just because QUIC extensions are essentially TLS extensions, and thus could show up there.

The primary goal for the fingerprint strings is enable exact matching between a protocol message and a database of fingerprint strings. Sort normalization is needed for that use case, which is why we developed the tls/1 format. We left the code in place to select the older tls/ format to support other use cases, such the one you are describing. There is more background info at https://hnull.org/2022/12/01/sorting-out-randomized-tls-fingerprints/.

Thanks for pointing out the issue of server responses depending on the extension order. It makes me wonder: does the Google/Android library avoid reordering some extensions? If so, that would introduce a recognizable behavior that could be used in fingerprinting. If not, it seems that some sessions might needlessly get rejected by the server.

Sort-normalization is used for both QUIC and TLS when the "tls/1" fingerprint format is in use. QUIC_Extension is included in the TLS fingerprint specification just because QUIC extensions are essentially TLS extensions, and thus could show up there.

Thanks for clearing that up. Do you agree that the current documentation you are responding to is vague here? If so I think it would be helpful to update it. This way people will clearly know that both tls/ and tls/1 support both QUIC and TLS extensions.

The primary goal for the fingerprint strings is enable exact matching between a protocol message and a database of fingerprint strings. Sort normalization is needed for that use case, which is why we developed the tls/1 format. We left the code in place to select the older tls/ format to support other use cases, such the one you are describing.

thats fine, but then it seems that the word "older" is misleading, as while its technically true, the "older" option is still valid and will remain so for the foreseeable future. both use cases are important, so in my opinion one is not "better" than the other, they have different use cases. It might be better to just call them:

• sorted/unsorted
• sorted/original

or similar. to me "older" is similar to "discontinued" or "obsolete".

Thanks for pointing out the issue of server responses depending on the extension order. It makes me wonder: does the Google/Android library avoid reordering some extensions?

I dont agree with your wording here. an implementation does not "avoid reordering". an implementation might just send the extensions in whatever order they were constructed in. "avoid reordering" suggests that the client took an active role against changing the order, when in all likelihood the extensions were just dumped into an array and sent as is. its reordering thats an active process, one that a client would purposefully have needed to implement.

If so, that would introduce a recognizable behavior that could be used in fingerprinting.

client and server are not randomizing or sorting here, as it would be wasted computation on both sides. at least in the case of android.googleapis.com, the server is spying on client requests in order to verify that the client hello matches that of an Android device. This is contrasted with the Google Chrome client, where the goal in regard to the client hello is to remain anonymous, at least in regard to naive fingerprinting implementations.