Invalid end of chunk (parse_chunk_size)

[caobug]

Hello, the end of the chunk should be ended by two sets of \r\n. but parse_chunk_size considers that one set is enough, which results in the inability to know the completion of message transmission in the TCP stream.

for example:

correct:

let buf = b"0\r\n\r\n";
assert_eq!(httparse::parse_chunk_size(buf), Ok(httparse::Status::Complete((3, 0))));

unexp:

let buf = b"0\r\n";
assert_eq!(httparse::parse_chunk_size(buf), Ok(httparse::Status::Partial));

let buf = b"0\r\n\r";
assert_eq!(httparse::parse_chunk_size(buf), Ok(httparse::Status::Partial));

[seanmonstar]

The unexpected ones look correct to me. Those are partial ends.

The top example looks wrong, though. It should be complete, and report that the end is at byte 5, no?

[caobug]

No. The function parse_chunk_size reads \r, then \n, then breaks and returns Complete.

Therefore correct only reads 3 bytes. The unexpected below will always read only the first 3 bytes.

https://github.com/seanmonstar/httparse/blob/a3c10fd3281d3b70becc1aa54a995b617f502045/src/lib.rs#L1300C36-L1300C36

[sw17ch]

tl;dr: Sean is right, and this issue can be closed.

For those who come across this issue in the future, Sean is right. The chunk header parser is only parsing to the end of the chunk header itself. The final chunk on transfer-encoding: chunked still has a chunk header, a body, and a chunk footer.

Let's look at a few examples.

The first is a normal chunk. Here the chunk size parsing will tell us that the chunk size is 4, and that the body starts 3 bytes in.

4\r\n <-- header
0123  <-- body
\r\n  <-- footer

And, indeed, parse_chunk_size says the body starts 3 bytes in, and runs for 4 bytes.

let buf = b"\
    4\r\n\
    0123\r\n\
    ";
assert_eq!(httparse::parse_chunk_size(buf), Ok(httparse::Status::Complete((3, 4))));

httparse::parse_chunk_size only tells you about the header. You need to consume the body and the footer yourself.

Now lets look at the final chunk:

0\r\n <-- header
\r\n  <-- footer

let buf = b"\
    0\r\n\
    \r\n\
    ";
assert_eq!(httparse::parse_chunk_size(buf), Ok(httparse::Status::Complete((3, 0))));

There's clearly no payload, but if there was, it would come at position 3. parse_chunk_size correctly tells us that the offset of the payload is at 3. However, RFC9112 tells us that the final chunk has special handling; namely, the final chunk can contain trailers. Consider:

0\r\n
first: a\r\n
second: b\r\n
third: c\r\n
\r\n

So, httparse::parse_chunk_size isn't failing to consume the whole final chunk. It's telling us that the trailers start at byte 3, and that we need to use the rules for parsing the trailer-section defined by RFC9112.

Why don't we always assume the trailers come at byte 3? That's because of chunk extensions (see RFC9112 section 7.1.1). Consider this example:

let buf = b"\
    0;a=1;b=2;c=3\r\n\
    first: a\r\n\
    second: b\r\n\
    third: c\r\n\
    \r\n\
    ";
assert_eq!(httparse::parse_chunk_size(buf), Ok(httparse::Status::Complete((15, 0))));

Here the chunk extensions mean that the final chunk header is more than 3 bytes, and that the trailers (if any exist) will start at offset 15 instead of 3.

So in short, there's nothing all that special about the final chunk header compared to other chunks except that it signals that we must now switch to processing trailers (followed by the end of the message) instead of body bytes.