WIP: add request config

[flavio]

This PR extends the req API to take a second, optional parameter. This parameter contains some tuning values such as:

• The list of extra CAs to be trusted. This is useful when interacting with servers that use self-signed certificates
• The client certificate and key. This is useful when the remote server requires mutual TLS authentication.
• And more things... :)

These settings are stored into a dedicates Struct which is modeled after what reqwest uses. The struct is controller by the Wasm client (this makes it possible to have a client specify its own identity, instead of having all the outgoing request from the host to use the same identity).
The request configuration is serialized to MsgPack by the Wasm guest, and is then deserialized by the Wasm host.

Currently only the Rust guest code has been extended to handle this serialization. The AssemblyScript code needs to be written. I can do that, but I would like to gather your feedback before doing that.

BTW, I picked MsgPack because we can reuse the waPC libraries made available for different programming languages to serialize structures into MsgPack.

Why did I do that?

During the holidays I played a bit with Wagi. I tried to interact with the Kubernetes API from within a Wasm module. To do that I used this crate to perform my requests.

As you probably know, the Kubernetes API server is exposed over a TLS endpoint that is often secured by a self-signed certificate.
Moreover, the client authentication can work in two ways: either through a token sent inside of the request headers or via client certificates. Obviously I wanted to use the client certificates, which was not doable before this PR.

The final outcome, which deserves a blog post on its own, works pretty well. I implemented a webhook endpoint using Wagi + Rust modules using this crate that can trigger actions inside of a Kubernetes cluster 😎

I had to make some small changes to Wagi too, I'll create a PR once I know if you're interested in this feature :)

[radu-matei]

Hi, @flavio, thanks a lot, this is awesome work!

First, I totally agree with the need for additional request options, such as CAs, certificates and keys, and potentially other things (trailers for HTTP/2 has also been suggested in #45).

The only reservation I have towards this is introducing serialization and deserialization for part of the data required by the library — although I understand there is no way around serialization with the current WITX syntax (which is partly why we haven't implemented this already).

To address this, we (the Bytecode Alliance and members of it) have been working towards a new interface format and bindings generator — https://github.com/bytecodealliance/wit-bindgen.
With the new format, we can model more complex data types, and avoid making de/serialization an implementation concern.

I have been experimenting rewriting this outbound HTTP library in the new WIT format — https://github.com/fermyon/wasi-experimental-toolkit/blob/main/wit/ephemeral/http-types.wit and https://github.com/fermyon/wasi-experimental-toolkit/blob/main/wit/ephemeral/wasi-outbound-http.wit, with the implementation (that closely mirrors this one) here https://github.com/fermyon/wasi-experimental-toolkit/blob/main/crates/http-wasmtime/src/lib.rs
(the goal for the interfaces linked in the repo above is to be proposed for standardization at some point).

How would you feel collaborating on adding this optional config for requests in the new WIT syntax?

Thanks again for the awesome work!

[flavio]

How would you feel collaborating on adding this optional config for requests in the new WIT syntax?

That's super exciting!

Give me some time to get acquainted with the links you posted. I'm all in!

[flavio]

Closing, the work has moved to https://github.com/fermyon/wasi-experimental-toolkit