http-compat: add support for convert between bhttp binary and http crate

[imlk0]

This PR introduces a new http-compat module that provides asynchronous streaming conversion between HTTP types from the http crate and BHTTP format. This enhancement allows seamless conversion between standard HTTP requests/responses and BHTTP binary format using asynchronous streams.

Note: this pull request is based on #74

Changes

• Added BhttpEncoder struct for converting http::Request/http::Response to BHTTP data stream (futures::prelude::Stream<Item = Res<Vec<u8>>>)
• Added BhttpDecoder struct for converting BHTTP binary data (futures::io::AsyncRead) back to http::Request/http::Response
• Introduced http-compat feature flag that enables this feature

Testing

• Added unit tests for request and response conversion based on examples in RFC 9292 Section 5.1 and Section 5.2
• You can run test with cargo test --features http-compat http_compat

Usage

With this change, users can now easily convert between standard HTTP types and BHTTP format:

// Encoding example
let request = http::Request::builder()
    .method("GET")
    .uri("https://example.com")
    .body(Full::new(Bytes::from("test")))
    .unwrap();

let stream = BhttpEncoder::from_request(request);
// Stream can be consumed to get BHTTP binary data
// For encoding response using
// let stream = BhttpEncoder::from_response(response);

// Decoding example
let bhttp_async_read = get_bhttp_bytes(); // Some BHTTP binary data
let decoder = BhttpDecoder::new(bhttp_async_read);
// Decoder can be used to get back http::Request or http::Response
let request_or_response = decoder.decode_message().await?;
match request_or_response {
    HttpRequestOrResponse::Request(request) => {
        // got http::Request
    }
    HttpRequestOrResponse::Response(response) => {
        // got http::Response
    }
}

[martinthomson]

Just some preliminary feedback. I find myself short on time today (as always), so it's not as in-depth as I'd like.

[martinthomson]

Suggested change

	pub enum HttpRequestOrResponse<R> {
	pub enum HttpMessage<R> {

How do you imagine someone using this type? Do they simply call HttpMessage::from(...)?

[imlk0]

The name HttpMessage seems clearer than before.

However, I don't think it makes sense to allow users to directly call HttpMessage::from(...) because that would require a BhttpBody, which should only be created through BhttpDecoder::decode_message(). To enforce this, I will make BhttpBody::new() private in a later commit.

[martinthomson]

Suggested change

	enum HttpRequestResponse<T> {
	enum HttpMessage<T> {

So the trick here is that you wrap and then implement functions to encode those messages into the underlying type. And T is the body type.

[imlk0]

Yes, the http crate uses a generic type T to represent the body type, and as long as T implements http_body::Body, I think this is a great design choice. Perhaps in the future we could refactor bhttp::Message into bhttp::Message<T>, which would allow us to avoid using content: Vec<u8> internally for storing content.

ohttp/bhttp/src/lib.rs

Lines 592 to 598 in c6131ac

	pub struct Message {
	informational: Vec<InformationalResponse>,
	control: ControlData,
	header: FieldSection,
	content: Vec<u8>,
	trailer: FieldSection,
	}

[martinthomson]

Suggested change

	// Modifyed example from Section 5.1 of RFC 9292 - Indeterminate-Length Binary Encoding of Request
	// Modified example from Section 5.1 of RFC 9292 - Indeterminate-Length Binary Encoding of Request

How modified?

[imlk0]

In the original bhttp example, the scheme is https, the authority is empty, and the path is /hello.txt. However, this is not allowed by the http crate (see source), which requires that if a scheme is present, the authority must also be present.

To make the test meaningful, I have modified the bhttp example to set a valid authority, ensuring it conforms to the HTTP URI specification.

I will add a comment describing the change later.

[martinthomson]

This seems pretty weak in terms of its utility as a test. I suggest that you have a set of known-answer tests for these. On the one hand, http::Request, on the other a byte array. Those can live in a common test module. This code can run down the list and check that the output matches the expectation.

[imlk0]

This is a great suggestion. It would make test expansion much more convenient. Do you have any ideas about geting more of such combinations? There are some in RFC 9292, but I've already used them all.

[imlk0]

Thank you for your feedback. I will update this PR later :)

[imlk0]

Updated based on your feedback and marked most reviews as resolved. Left a few open for further discussion, would appreciate your thoughts!

[martinthomson]

@imlk0, I'm sorry, but I didn't realize that merging the other PR would close this. And I can't reopen it. Can you do that? I think it needs a new pull request. (I'm still considering taking this code; it's good, but I need to do a bit more testing on my own to get more confident in it.)

[imlk0]

Hi @martinthomson

Sorry for the long delay in getting back to this. I’ve reopened the PR and rebased it onto the latest master (#87). I’ve also cleaned up and squashed some of the earlier commits for a clearer history.

Based on further testing and use in my side, I’ve incorporated several improvements to enhance stability and functionality:

• 4bc4d63 fix(http-compat): remove default value for uri when convert between bhttp and http (Kun Lai)
• 321f202 fix(ohttp): change nonce array length from 16 to 32 to fix panic (Kun Lai)
• 0106d0d feat(http-compat): allow creating server response while client request is consuming (Kun Lai)

I’ve been using this branch in my own project to encrypt both HTTP requests and responses (including SSE streaming) for vLLM inference, and it’s been working well.

I’m still committed to helping move this forward. please let me know if you'd like any further adjustments or have additional feedback :).

Thanks!