Contribute to http2jp/hpack-test-case

[dshafik]

Hi,

I think it would be great if this library could contribute results to the shared hpack test result repo: https://github.com/http2jp/hpack-test-case

This also provides a large set of example data to run tests against. I plan to contribute hyphper/h2-hpack results to it, and would love to compare.

[dshafik]

Related to this, I see that, one of the test cases (#3 here) causes different output in python-hyper/hpack, although it can parse it back just fine:

hexlify(bytes(e.encode([(':method', 'GET'), (':scheme', 'http'), (':authority', 'k.yimg.jp'), (':path', '/images/top/sp2/cmn/logo-ns-130528.png')])))

Results in the following hex:

'82864187eabfa35332fd2b449b60d48e62a1849eb611589825353141e63ad52160b206c4f2f5d537'
-----------------------^

Notice the \x44 at the indicated location, in all other implementations, this is \x04. This implementation is able to decode both, with no difference in result. I believe this is the prefix, as created here.

I'm not sure why this is different, I attempted to look at some other hpack implementations but none of them are easy to read like this one. I have the same result in hyphper/h2-hpack not surprisingly, causing this, and I believe most of the other tests to fail.

[tatsuhiro-t]

About 0x04 vs 0x44, it is "Literal Header Field without Indexing" vs "Literal Header Field with Incremental Indexing". At least for libnghttp2, it does not perform indexing for :path header field, this is because :path is usually not repeating much, and with our data, not indexing it gives us better compression ratio. You know, path could be very long these days.
It is completely fine to perform indexing of :path.

[dshafik]

@tatsuhiro-t can you point me to the code for that behavior?

[Lukasa]

@dshafik Agreed, it'd be good to provide this data. I'm going to shove this on my to-do list (I'm only just back from holiday so I'm triaging everything that came up).

[tatsuhiro-t]

@dshafik nghttp2? https://github.com/nghttp2/nghttp2/blob/3c3267ea7d3bc3cb5baf228fe45331bee15324c7/lib/nghttp2_hd.c#L1348

[Lukasa]

Alrighty, I've submitted my results upstream as http2jp/hpack-test-case#23.

[dshafik]

These have been merged, so I'm closing out the bug. Thanks @Lukasa!