-
Notifications
You must be signed in to change notification settings - Fork 311
libp2p + HTTP #477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
libp2p + HTTP #477
Changes from 2 commits
ecdc680
d52be51
3b5ee42
cdc729e
d04725f
1c777be
c7deac5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| # libp2p over HTTP <!-- omit in toc --> | ||
|
|
||
| | Lifecycle Stage | Maturity | Status | Latest Revision | | ||
| | --------------- | ------------- | ------ | --------------- | | ||
| | 1A | Working Draft | Active | r0, 2022-11-10 | | ||
|
|
||
| Authors: [@marcopolo] | ||
|
|
||
| Interest Group: [@marcopolo], [@mxinden], [@marten-seemann] | ||
|
|
||
| [@marcopolo]: https://github.com/mxinden | ||
| [@mxinden]: https://github.com/mxinden | ||
| [@marten-seemann]: https://github.com/marten-seemann | ||
|
|
||
| # Table of Contents <!-- omit in toc --> | ||
| - [Context](#context) | ||
| - [Why not two separate stacks?](#why-not-two-separate-stacks) | ||
| - [Why HTTP rather than a custom request/response protocol?](#why-http-rather-than-a-custom-requestresponse-protocol) | ||
| - [Implementation](#implementation) | ||
| - [HTTP over libp2p streams](#http-over-libp2p-streams) | ||
| - [libp2p over plain HTTPS](#libp2p-over-plain-https) | ||
| - [Choosing between libp2p streams vs plain HTTPS](#choosing-between-libp2p-streams-vs-plain-https) | ||
| - [Implementation recommendations](#implementation-recommendations) | ||
| - [Example – Go](#example--go) | ||
| - [Prior art](#prior-art) | ||
|
|
||
| # Context | ||
|
|
||
| HTTP is everywhere. Especially in CDNs, cloud offerings, and caches. | ||
|
|
||
| HTTP on libp2p and libp2p on HTTP are both commonly requested features. This has | ||
| come up recently at IPFS Camp and especially in the [data transfer track]. One | ||
| aspect of the discussion makes it seem like you can use HTTP _OR_ use libp2p, | ||
| but that isn't the case. Before this spec you could use the HTTP protocol on top | ||
| of a libp2p stream (with little to no extra cost). And this spec outlines how to use libp2p _on top of_ HTTP. | ||
|
|
||
| This spec defines a new libp2p abstraction for stateless request/response | ||
| protocols. This abstraction is notably nothing new, it is simply HTTP. Being | ||
| HTTP, This abstraction can run over a plain TCP+TLS HTTP (henceforth referred to | ||
| as _plain https_) or on top of a libp2p stream. | ||
|
|
||
| ## Why not two separate stacks? | ||
|
|
||
| Having libp2p as the abstraction over _how_ the HTTP request gets sent gives developers a lot of benefits for free, such as: | ||
|
|
||
| 1. NAT traversal: You can make an HTTP request to a peer that's behind a NAT. | ||
| 1. Fewer connections: If you already have a libp2p connection, we can use use that to create a stream for the HTTP request. So that HTTP request will be much faster (You don't have to pay the two round trips to establish the connection) | ||
|
MarcoPolo marked this conversation as resolved.
Outdated
|
||
| 1. Allows JS clients to make HTTPS requests to _any_ peer via WebTransport. | ||
| 1. Allows more reuse of the protocol logic, just like how applications can integrate GossipSub, bitswap, graphsync, and Kademlia. | ||
| 1. You get mutual authentication of peer IDs automatically. | ||
|
|
||
| Get the benefits of libp2p (webtransport, fewer connections, webrtc, nat traversal) | ||
|
MarcoPolo marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Why HTTP rather than a custom request/response protocol? | ||
|
|
||
| HTTP has been around for 30+ years, and it isn't going anywhere. Developers are already very familiar with it. There's is no need to reinvent the wheel here. | ||
|
|
||
| # Implementation | ||
|
|
||
| ## HTTP over libp2p streams | ||
|
|
||
| If we have an existing libp2p connection that supports streams, we can run the HTTP protocol as follows: | ||
|
|
||
| Client: | ||
| 1. Open a new stream to the target peer. | ||
| 1. Negotiate the `/libp2p-http` protocol. | ||
| 1. Use this stream for HTTP. (i.e. start sending the request) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this HTTP 1.1?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. https://datatracker.ietf.org/doc/rfc9292/ could be useful as the wire format. |
||
| 1. Close the write side when finished uploading the HTTP request. | ||
| 1. Close the stream when the response is received. | ||
|
|
||
| Server: | ||
| 1. Register the `/libp2p-http` protocol. | ||
| 1. One a new stream speaking `/libp2p-http` pass the stream to an HTTP handler. | ||
|
MarcoPolo marked this conversation as resolved.
Outdated
|
||
| 1. Close the stream when finished uploading the request. | ||
|
MarcoPolo marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## libp2p over plain HTTPS | ||
|
|
||
| This is nothing more than a thin wrapper over standard HTTP. The only thing | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This might be slightly OT but has the https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Upgrade GET /index.html HTTP/1.1
Host: www.example.com
Connection: upgrade
Upgrade: multistream-select/1.0.0Once confirmed, we can then negotiate any other protocol on top, i.e. yamux, noise, etc. This could be a neat way for establishing a libp2p connection from the browser, assuming the peer we want to reach exposes an HTTP endpoint we can use to trigger the upgrade.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Assuming we define a corresponding "libp2p over http" multiaddress protocol, we can build a
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe: Note that
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This sounds like reinventing WebSocket.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. WebSockets have a framing overhead though whereas unless I am missing something, using |
||
| libp2p should do here is ensure that we verify the peer's TLS certificate as | ||
| defined by the [tls spec](../tls/tls.md). This SHOULD be interoperable with standard HTTP clients who pass a correct TLS cert. For example curl should work fine: | ||
|
|
||
| ``` | ||
| $ curl --insecure --cert ./client.cert --key ./client.key https://127.0.0.1:9561/echo -d "Hello World" | ||
|
|
||
| Hello World | ||
| ``` | ||
|
|
||
| ## Choosing between libp2p streams vs plain HTTPS | ||
|
|
||
| Implementations SHOULD choose a libp2p stream if an existing libp2p connection | ||
| is available. If there is an existing HTTP connection, then implementations | ||
| SHOULD use that connection rather than starting a new libp2p connection. If | ||
| there is no connection implementations may choose either to create a new HTTP | ||
| connection or a libp2p connection or expose this as an option to users. | ||
|
|
||
| # Implementation recommendations | ||
|
|
||
| Each implementation should decide how this works, but the general recommendations are: | ||
|
|
||
| 1. Make this look and feel like a normal HTTP client and server. There's no | ||
| benefit of doing things differently here, and the familiarity will let people | ||
| build things faster. | ||
|
|
||
| 1. Aim to make the returned libp2p+HTTP objects interop with the general HTTP ecosystem of the language. | ||
|
|
||
| ## Example – Go | ||
|
|
||
| We create a host as normal, but enable HTTP: | ||
| ``` | ||
| h, err := libp2p.New(libp2p.WithHTTP( | ||
| HTTPConfig: HTTPConfig{ | ||
| EnableHTTP: true, | ||
| // Enable | ||
| HTTPServerAddr: multiaddr.StringCast("/ip4/127.0.0.1/tcp/9561/tls/http"), | ||
| })) | ||
| ``` | ||
|
|
||
| We can define HTTP Handlers using standard types: | ||
| ``` | ||
| h1.SetHTTPHandler("/echo", func(peer peer.ID, w http.ResponseWriter, r *http.Request) { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be really nice if we could reuse the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like this idea! |
||
| w.WriteHeader(200) | ||
| io.Copy(w, r.Body) | ||
| }) | ||
| ``` | ||
|
|
||
| We can create a client that accepts standard types | ||
| ``` | ||
| // client is a standard http.Client | ||
| client, err := h2.NewHTTPClient(h1.ID()) | ||
| require.NoError(t, err) | ||
|
|
||
| resp, err := client.Post("/echo", "application/octet-stream", bytes.NewReader([]byte("Hello World"))) | ||
| ``` | ||
|
|
||
| For more details see the implementation [PR](https://github.com/libp2p/go-libp2p/pull/1874). | ||
|
|
||
| # Prior art | ||
|
|
||
| - rust-libp2p's request-response protocol: https://github.com/libp2p/rust-libp2p/tree/master/protocols/request-response | ||
| - go-libp2p's [go-libp2p-http] | ||
|
|
||
| [data transfer track]: (https://youtube.com/watch?v=VRn_U8ytvok&feature=share&si=EMSIkaIECMiOmarE6JChQQ) | ||
| [rust-libp2p request-response protocol]: (https://github.com/libp2p/rust-libp2p/tree/master/protocols/request-response) | ||
| [go-libp2p-http]: (https://github.com/libp2p/go-libp2p-http) | ||
Uh oh!
There was an error while loading. Please reload this page.