diff --git a/protocol/README.md b/protocol/README.md index e79222a84..ab31eac4e 100644 --- a/protocol/README.md +++ b/protocol/README.md @@ -1,22 +1,32 @@ -# IPFS Protocol Spec (WIP!) +IPFS Protocol Spec +================== -Authors: [@jbenet](http://github.com/jbenet) +> **This spec is an Work In Progress (WIP)** + +Authors: + +- [Juan Benet](https://github.com/jbenet) +- [David Dias](https://github.com/diasdavid) Reviewers: * * * -This [spec](../) document defines the IPFS protocol stack, the subsystems, the +This spec document defines the IPFS protocol stack, the subsystems, the interfaces, and how it all fits together. It delegates non-interface details to other specs as much as possible. This is meant as a top-level view of the protocol and how the system fits together. - Note, this document is not meant to be an introduction of the concepts in IPFS and is not recommended as a first pass to understanding how IPFS works. For that, please refer to the [IPFS paper](http://static.benet.ai/t/ipfs.pdf). -## IPFS and the Merkle DAG +# Index + +- []() +- []() + +## 1. IPFS and the Merkle DAG At the heart of IPFS is the MerkleDAG, a directed acyclic graph whose links are hashes. This gives all objects in IPFS useful properties: @@ -41,7 +51,7 @@ publish, distribute, serve, and download merkledags. It is the authenticated, decentralized, permanent web. -## Nodes and Network Model +## 2. Nodes and Network Model The IPFS network uses PKI based identity. An "ipfs node" is a program that can find, publish, and replicate merkledag objects. Its identity is defined @@ -54,7 +64,7 @@ nodeID := multihash(publicKey) TODO: constraints on keygen. -### multihash and upgradeable hashing +### 2.1 multihash and upgradeable hashing All hashes in ipfs are encoded with [multihash](https://github.com/jbenet/multihash/), a self-describing hash @@ -75,7 +85,7 @@ sha3 ``` -## The Stack +## 3. The Stack IPFS has a stack of modular protocols. Each layer may have multiple implementations, all in different modules. This spec will only address the @@ -94,7 +104,7 @@ IPFS has five layers: These are briefly described bottom-up. -### Network -- connecting to peers +### [3.1 Network](network) The **network** provides point-to-point transports (reliable and unreliable) between any two IPFS nodes in the network. It handles: @@ -105,7 +115,7 @@ between any two IPFS nodes in the network. It handles: See more in the [network spec](network). -### Routing -- finding peers and data +### [3.2 Routing -- finding peers and data](routing) The IPFS **Routing** layer serves two important purposes: - **peer routing** -- to find other nodes @@ -122,9 +132,9 @@ of implementations. For example: to one of a set of supernodes. This is roughly like federated routing. - **dns:** ipfs routing could even happen over dns. -See more in the routing spec (TODO). +See more in the [routing spec](https://github.com/ipfs/specs/tree/master/protocol/routing). -### Block Exchange -- transfering content-addressed data +### [3.3 Block Exchange -- transfering content-addressed data](exchange) The IPFS **Block Exchange** takes care of negotiating bulk data transfers. Once nodes know each other -- and are connected -- the exchange protocols @@ -137,7 +147,7 @@ of implementations. For example: of BitTorrent to work with arbitrary (and not known apriori) DAGs. - **HTTP:** a simple exchange can be implemented with HTTP clients and servers. -### Merkledag -- making sense of data +### [3.4. Merkledag -- making sense of data](../merkledag) [As discussed above](#IPFS-and-the-Merkle-DAG), the IPFS **merkledag** is the datastructure at the heart of IPFS. It is an @@ -170,7 +180,7 @@ on top of the merkledag, such as: See more in the merkledag spec (TODO). -### Merkledag Paths +### [3.4.1 Merkledag Paths](../merkledag) The merkledag is enough to resolve paths: @@ -186,7 +196,7 @@ See more in the path resolution spec (TODO). ![](../media/ipfs-resolve/ipfs-resolve.gif) -### Naming -- PKI namespace and mutable pointers +### [3.5 Naming -- PKI namespace and mutable pointers]() IPFS is mostly concerned with content-addressed data, which by nature is immutable: changing an object would change its hash -- and thus its @@ -209,7 +219,7 @@ See more in the namin spec (TODO). -## Applications and Datastructures -- on top of IPFS +## [4. Applications and Datastructures -- on top of IPFS]() The stack described so far is enough to represent arbitrary datastructures and replicate them accross the internet. It is also enough to build and @@ -222,7 +232,7 @@ them to the rest of the world using any of the tools that understand IPFS. See more in the datastructures and applications specs (TODO). -### unixfs -- representing traditional files +### [4.1 unixfs -- representing traditional files]() The unix filesystem abstractions -- files and directories -- are the main way people conceive of files in the internet. In IPFS, `unixfs` is a datastructure @@ -235,7 +245,7 @@ to carry over information like: See more in the unixfs spec (TODO). -## Lifetime of fetching an object. +## [5 Lifetime of fetching an object.]() Suppose we ask an IPFS node to retrieve @@ -253,11 +263,7 @@ Then, the IPFS node resolves the components. The first component in an `/ipfs/...` path is always a multihash. The rest are names of links, to be resolved into multihashes. - - - - -## IPFS User Interfaces +## [6 IPFS User Interfaces]() IPFS is not just a protocol. It is also a toolset. IPFS implementations include various tools for working with the merkledag, how to publish @@ -271,9 +277,10 @@ design and implementation. Examples: - The IPFS libs - implementations in various languages - The IPFS gateways - nodes in the internet that serve HTTP over IPFS -## ~~WIP~~ +* * * + +### WIP Stack Dump: -WIP Stack Dump: - How the layers fit together - How they call on each other - Mention all the ports diff --git a/protocol/network/README.md b/protocol/network/README.md index a5141877f..42bd2a6cd 100644 --- a/protocol/network/README.md +++ b/protocol/network/README.md @@ -1,20 +1,53 @@ -# IPFS Protocol Network Spec or `libp2p` +IPFS Protocol Network Spec +========================= -Authors: [Juan Benet](http://github.com/jbenet) +Authors: +- [Juan Benet](https://github.com/jbenet) +- [David Dias](https://github.com/diasdavid) Reviewers: -TODOS: -- incorporate peer-routing, as discussed in https://github.com/ipfs/specs/issues/1 - * * * -This [spec](../../) describes the IPFS network protocol. The network layer -provides point-to-point transports (reliable and unreliable) between any two -IPFS nodes in the network. - - -## Supports +# Abstract + +This describes the IPFS network protocol. The network layer provides point-to-point transports (reliable and unreliable) between any two IPFS nodes in the network. + +This document defines the spec implemented in libp2p. + +# Status of this spec + +> This spec is a Work In Progress (WIP) + +# Table of Contents + +- [1 Introduction and Goals]() +- [2 Requirements]() + - [2.1 NAT traversal] () + - [2.2 Relay]() + - [2.3 Ecryption]() + - [2.4 Transport Agnostic]() + - [2.5 Multi-Multiplexing]() +- [3 Datastructures]() +- [4 Interface]() + - [4.1 Client Interface]() + - [4.2 Protocol Interface]() +- [5 Properties]() + - [5.1 Communication Model - Streams]() + - [5.2 Ports - Constrained Entrypoints]() + - [5.3 Transport Protocol]() + - [5.4 Non-IP Networks]() + - [5.5 On the wire]() + - [5.5.1 Protocol-Multiplexing]() + - [5.5.2 multistream - self-describing protocol stream]() + - [5.5.3 multistream-selector - self-describing protocol stream selector]() + - [5.5.4 Stream Multiplexing]() + - [5.5.5 Portable Encodings]() +- [6 Software Stack]() +- [7 Implementation Details]() +- [References]() + +## 1. Introduction and Goals It SHOULD support: - [NAT traversal](#NAT-traversal) @@ -23,35 +56,21 @@ It SHOULD support: - [Multiple Transports](#Transport-Agnostic) - [Multi-Multiplexing](#Multi-multiplexing) +## 2. Requirements -### NAT traversal +### 2.1 NAT traversal -Network Address Translation is ubiquitous in the internet. Not only are most -consumer devices behind many layers of NATs, but most datacenter nodes are -often behind NAT for security or virtualization reasons. As we move into -containerized deployments, this is getting worse. IPFS implementations SHOULD -provide a way to traverse NATs, otherwise it is likely that operation will be -affected. Even nodes meant to run with real IP addresses must implement NAT -traversal techniques, as they may need to establish connections to peers -behind NAT. +Network Address Translation is ubiquitous in the internet. Not only are most consumer devices behind many layers of NATs, but most datacenter nodes are often behind NAT for security or virtualization reasons. As we move into containerized deployments, this is getting worse. IPFS implementations SHOULD provide a way to traverse NATs, otherwise it is likely that operation will be affected. Even nodes meant to run with real IP addresses must implement NAT traversal techniques, as they may need to establish connections to peers behind NAT. -IPFS accomplishes full NAT traversal using an ICE-like protocol. It is not -exactly ICE, as ipfs networks provide the possibility of relaying communications over the IPFS protocol itself, for coordinating hole- -punching or even relaying communication. +IPFS accomplishes full NAT traversal using an ICE-like protocol. It is not exactly ICE, as ipfs networks provide the possibility of relaying communications over the IPFS protocol itself, for coordinating hole-punching or even relaying communication. -It is recommended that implementations use one of the many NAT traversal -libraries available, such as `libnice`, `libwebrtc`, or `natty`. However, -NAT traversal must be interoperable. +It is recommended that implementations use one of the many NAT traversal libraries available, such as `libnice`, `libwebrtc`, or `natty`. However, NAT traversal must be interoperable. -### Relay +### 2.2 Relay -Unfortunately, due to symmetric NATs, container and VM NATs, and other -impossible-to-bypass NATs, IPFS MUST fallback to relaying communication -to establish a full connectivity graph. To be complete, implementations -MUST support relay, though it SHOULD be optional and able to be turned -off by end users. +Unfortunately, due to symmetric NATs, container and VM NATs, and other impossible-to-bypass NATs, IPFS MUST fallback to relaying communication to establish a full connectivity graph. To be complete, implementations MUST support relay, though it SHOULD be optional and able to be turned off by end users. -### Encryption +### 2.3 Encryption Communications on IPFS may be: @@ -69,23 +88,13 @@ We recommend that: IPFS uses cyphersuites like TLS. -**NOTE:** we do not use TLS directly, because we do not want the CA system -baggage. Most TLS implementations are very big. Since the IPFS model begins -with keys, IPFS only needs to apply ciphers. This is a minimal portion of the -whole TLS standard. +**NOTE:** we do not use TLS directly, because we do not want the CA system baggage. Most TLS implementations are very big. Since the IPFS model begins with keys, IPFS only needs to apply ciphers. This is a minimal portion of the whole TLS standard. -### Transport Agnostic +### 2.4 Transport Agnostic -IPFS is transport agnostic, so it can run over any transport protocol. It does -not even depend on IP; it may run on top of NDN, XIA, and other new internet -architectures. +IPFS is transport agnostic, so it can run over any transport protocol. It does not even depend on IP; it may run on top of NDN, XIA, and other new internet architectures. -In order to reason about possible transports, IPFS uses -[multiaddr](https://github.com/jbenet/multiaddr), a self-describing addressing -format. This makes it possible for IPFS to treat addresses opaquely everywhere -in the system, and have support various transport protocols in the network -layer. The actual format of addresses in IPFS is `ipfs-addr`, a multiaddr that -ends with an ipfs nodeid. For example, these are all valid `ipfs-addrs`: +In order to reason about possible transports, IPFS uses [multiaddr](https://github.com/jbenet/multiaddr), a self-describing addressing format. This makes it possible for IPFS to treat addresses opaquely everywhere in the system, and have support various transport protocols in the network layer. The actual format of addresses in IPFS is `ipfs-addr`, a multiaddr that ends with an ipfs nodeid. For example, these are all valid `ipfs-addrs`: ``` # ipfs over tcp over ipv6 (typical tcp) @@ -104,19 +113,13 @@ ends with an ipfs nodeid. For example, these are all valid `ipfs-addrs`: /ether/ac:fd:ec:0b:7c:fe/ipfs/QmYJyUMAcXEw1b5bFfbBbzYu5wyyjLMRHXGUkCXpag74Fu ``` -**Note:** at this time, no unreliable implementations exist. The protocol's -interface for defining and using unreliable transport has not been defined. +**Note:** at this time, no unreliable implementations exist. The protocol's interface for defining and using unreliable transport has not been defined. **TODO:** define how unreliable transport would work. base it on webrtc. +### 2.5 Multi-Multiplexing -### Multi-Multiplexing - -The IPFS Protocol is a collection of multiple protocols available at the same -IPFS Node. In order to conserve resources, and to make connectivity easier, -the IPFS network layer can perform all its operations through a single TCP or -UDP port, depending on the transports used. IPFS can multiplex its many -protocols through point-to-point connections. This multiplexing is for both +The IPFS Protocol is a collection of multiple protocols available at the same IPFS Node. In order to conserve resources, and to make connectivity easier, the IPFS network layer can perform all its operations through a single TCP or UDP port, depending on the transports used. IPFS can multiplex its many protocols through point-to-point connections. This multiplexing is for both reliable streams and unreliable datagrams. IPFS is pragmatic. It seeks to be usable in as many settings as possible, to @@ -154,26 +157,27 @@ ensures that complex user or application constraints do not rule out IPFS as an option. -## Datastructures +## 3. Datastructures The network protocol deals with these datastructures: - a `PrivateKey`, the private key of a node. - a `PublicKey`, the public key of a node. - a `PeerID`, a hash of a node's public key. -- a `Node`[*], has a PeerID, and open connections to other `Nodes`. +- a `Node`[1], has a PeerID, and open connections to other `Nodes`. - a `Connection`, a point-to-point link between two Nodes (muxes 1 or more streams) - a `Stream`, a duplex message channel. -[*] currently called `PeerHost` in go-ipfs. +[1] currently called `PeerHost` in go-ipfs. + +## 4. Interface -## Interface +The network protocol's interface has two parts:A -The network protocol's interface has two parts: 1. the _client interface_, for clients (e.g. higher layers of IPFS) 2. the _service interface_, for remote peers (e.g. other IPFS nodes) -### Client Interface +### 4.1 Client Interface The **Client Interface** is exposed to the higher layers of IPFS. It is the entry point for other parts to open + handle streams. @@ -221,7 +225,7 @@ type StreamHandler func (Stream) TODO: incorporate unreliable message / packet streams. -### Protocol Interface +### 4.2 Protocol Interface The network protocol consists of: @@ -246,9 +250,9 @@ for interoperability. These are the default (TODO: unreliable transport) -## Properties +## 5 Properties -### Communication Model - Streams +### 5.1 Communication Model - Streams The Network layer handles all the problems of connecting to a peer, and exposes simple bidirectional streams. Users can both open a new stream @@ -282,7 +286,7 @@ stream.Read(buf2) // read what was sent back fmt.Println(buf2) // print what was sent back ``` -### Ports - Constrained Entrypoints +### 5.2 Ports - Constrained Entrypoints In the internet of 2015, we have a processing model where a program may be running without the ability to open multiple -- or even single -- network @@ -299,7 +303,7 @@ through Websockets or WebRTC. In a sense, the role of the TCP/UDP network stack -- i.e. multiplexing applications and connections -- may now be forced to happen at the application level. -### Transport Protocols +### 5.3 Transport Protocols IPFS is transport agnostic. It can run on any transport protocol. The `ipfs-addr` format (which is an ipfs-specific @@ -334,7 +338,7 @@ Some of the transport protocols we will be using: - Websockets - TCP Remy -### Non-IP Networks +### 5.4 Non-IP Networks Efforts like [NDN](http://named-data.net) and [XIA](http://www.cs.cmu.edu/~xia/) are new architectures for the internet, @@ -343,3 +347,173 @@ will be able to operate on top of these architectures trivially, as there is no assumptions made about the network stack in the protocol. Implementations will likley need to change, but changing implementations is vastly easier than changing protocols. + +### 5.5 On the wire + +We have the **hard constraint** of making IPFS work across _any_ duplex stream (an outgoing and an incoming stream pair, any arbitrary connection) and work on _any_ platform. + +To make this work, IPFS has to solve a few problems: + +- [Protocol Multiplexing](#protocol-multiplexing) - running multiple protocols over the same stream + - [multistream](#multistream) - self-describing protocol streams + - [multistream-select](#multistream-select) - a self-describing protocol selector + - [Stream Multiplexing](#stream-multiplexing) - running many independent streams over the same wire. +- [Portable Encodings](#portable-encodings) - using portable serialization formats +- [Secure Communications](#secure-communication) - using ciphersuites to establish security and privacy (like TLS). + +#### 5.5.1 Protocol-Multiplexing + +Protocol Multiplexing means running multiple different protocols over the same stream. This could happen sequentially (one after the other), or concurrently (at the same time, with their messages interleaved). We achieve protocol multiplexing using three pieces: + +- [multistream](#multistream) - self-describing protocol streams +- [multistream-select](#multistream-select) - a self-describing protocol selector +- [Stream Multiplexing](#stream-multiplexing) - running many independent streams over the same wire. + +#### 5.5.2 multistream - self-describing protocol stream + +[multistream](https://github.com/jbenet/multistream) is a self-describing protocol stream format. It is extremely simple. Its goal is to define a way to add headers to protocols that describe the protocol itself. It is sort of like adding versions to a protocol, but being extremely explicit. + +For example: + +``` +/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 + + +... +``` + +#### 5.5.3 multistream-selector - self-describing protocol stream selector + +[multistream-select](https://github.com/jbenet/multistream/tree/master/multistream-select) is a simple [multistream](https://github.com/jbenet/multistream) protocol that allows listing and selecting other protocols. This means that Protomux has a list of registered protocols, listens for one, and then _nests_ (or upgrades) the connection to speak the registered protocol. This takes direct advantage of multistream: it enables interleaving multiple protocols, as well as inspecting what protocols might be spoken by the remote endpoint. + +For example: + +``` +/ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 +/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 + + +... +``` + +#### 5.5.4 Stream Multiplexing + +Stream Multiplexing is the process of multiplexing (or combining) many different streams into a single one. This is a complicated subject because it enables protocols to run concurrently over the same wire. And all sorts of notions regarding fairness, flow control, head-of-line blocking, etc. start affecting the protocols. In practice, stream multiplexing is well understood and there are many stream multiplexing protocols. To name a few: + +- HTTP/2 +- SPDY +- QUIC +- SSH + +IPFS nodes are free to support whatever stream multiplexors they wish, on top of the default one. The default one is there to enable even the simplest of nodes to speak multiple protocols at once. The default multiplexor will be HTTP/2 (or maybe QUIC?), but implementations for it are sparse, so we are beginning with SPDY. We simply select which protocol to use with a multistream header. + +For example: + +``` +/ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 +/ipfs/Qmb4d8ZLuqnnVptqTxwqt3aFqgPYruAbfeksvRV1Ds8Gri/spdy/3 + +/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 + + + +/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-bitswap/0.3.0 + + + + + + + + + + + + +... +``` + +#### 5.5.5 Portable Encodings + +In order to be ubiquitous, we _must_ use hyper-portable format encodings, those that are easy to use in various other platforms. Ideally these encodings are well-tested in the wild, and widely used. There may be cases where multiple encodings have to be supported (and hence we may need a [multicodec](https://github.com/jbenet/multicodec) self-describing encoding), but this has so far not been needed. + +For now, we use [protobuf](https://github.com/google/protobuf) for all protocol messages exclusively, but other good candidates are [capnp](https://capnproto.org), [bson](http://bsonspec.org/), [ubjson](http://ubjson.org/). + +## 6 Software Stack + +### 6.1 Overview + +### 6.2 Discovery + +goal: find more peers, keep routing table fresh (if Kad-Router is not being used, discovery doens't necessary has a use) + +### 6.3 Peer Routing + +goal: get ref to other peers, that then can be used by swarm to open a stream. Also is free to open streams to other peers to traverse the DHT + +### 6.4 Swarm (aka Connectivity) + +goal: open stream, NAT traversal, Relay + +~~The network is abstracted through the swarm which presents a simplified interface for the remaining layers to have access to the network. This interface should look like:~~ + +- `.openStream(peer, protocol)` - peer should contain the ID of the peer and its respective multiaddrs known. +- `.registerHandler(protocol, handlerFunc)` - enable a protocol to be registered, so that another peer can open a stream to talk with us to that specific protocol +- `.listen()` - to start listening for incoming connections and therefore opening of streams + +The following figure represents how the network level pieces, are tied together: + +``` +┌ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ┌───────────┐ + mounted │ mounted │ mounted ││Identify │ +│protocol │protocol │protocol │(mounted │ + 1 │ 2 │ ... ││ protocol) │ +└ ─ ─ ─ ─ └ ─ ─ ─ ─ └ ─ ─ ─ ─ └───────────┘ +┌─────────────────────────────────────────┐ +│ swarm │ +└─────────────────────────────────────────┘ +┌─────────────────────────────────────────┐ +│ connection │ +└─────────────────────────────────────────┘ +┌───────────────┐┌───────────┐┌───────────┐ +│Transport ││multistream││ stream │ +│(TCP, UDP, etc)││ ││ muxer │ +└───────────────┘└───────────┘│┌ ─ ─ ─ ─ ┐│ + │ spdy │ + │└ ─ ─ ─ ─ ┘│ + │┌ ─ ─ ─ ─ ┐│ + │ multiplex │ + │└ ─ ─ ─ ─ ┘│ + │┌ ─ ─ ─ ─ ┐│ + │ QUIC │ + │└ ─ ─ ─ ─ ┘│ + │┌ ─ ─ ─ ─ ┐│ + │ others │ + │└ ─ ─ ─ ─ ┘│ + └───────────┘ +``` + +**Identify** is one of the protocols mounted on top of swarm, our Connection handler, however, it follows and respects the same pattern as any other protocol when it comes to mounting it on top of swarm. Identify enables us to trade listenAddrs and observedAddrs between peers, this is crucial for the working of IPFS, since every socket open implements REUSEPORT, an observedAddr by another peer can enable a third peer to connect to us, since the port will be already open and redirect to us on a NAT. + +The stream muxer must implement the interface offered by [abstract-stream-muxer](https://github.com/diasdavid/abstract-stream-muxer). + +Every socket open (through the transport chosen), is "multistream'ed" into the stream muxer used, once a stream muxer connection + +### 6.5 libp2p + +## 7 Implementation Details + +### 7.1 Discovery + +### 7.2 Peer Routing + +### 7.3 Swarm + +Identify stream requests should be issued by the listenner as soon as it receives a valid connection, otherwise the listenner won't be able to identify who is that stream comming, disabling its ability for connection reuse. Identify is responsible for 'tagging' the incomming connection on swarm with the right Id. + +A peer only updates its own multiaddrs list with observedAddrs if it receives the same observedAddr twice, avoiding addr explosion (a phenomenon that happens when both peers are behind symmetric NAT). + +### 7.4 libp2p + + +## References diff --git a/protocol/network/figs/overview.monopic b/protocol/network/figs/overview.monopic new file mode 100644 index 000000000..c54bb898f Binary files /dev/null and b/protocol/network/figs/overview.monopic differ diff --git a/protocol/network/figs/overview.txt b/protocol/network/figs/overview.txt new file mode 100644 index 000000000..ecc37f9a6 --- /dev/null +++ b/protocol/network/figs/overview.txt @@ -0,0 +1,27 @@ +┌ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ┌───────────┐ + mounted │ mounted │ mounted ││Identify │ +│protocol │protocol │protocol │(mounted │ + 1 │ 2 │ ... ││ protocol) │ +└ ─ ─ ─ ─ └ ─ ─ ─ ─ └ ─ ─ ─ ─ └───────────┘ +┌─────────────────────────────────────────┐ +│ swarm │ +└─────────────────────────────────────────┘ +┌─────────────────────────────────────────┐ +│ connection │ +└─────────────────────────────────────────┘ +┌───────────────┐┌───────────┐┌───────────┐ +│Transport ││multistream││ stream │ +│(TCP, UDP, etc)││ ││ muxer │ +└───────────────┘└───────────┘│┌ ─ ─ ─ ─ ┐│ + │ spdy │ + │└ ─ ─ ─ ─ ┘│ + │┌ ─ ─ ─ ─ ┐│ + │ multiplex │ + │└ ─ ─ ─ ─ ┘│ + │┌ ─ ─ ─ ─ ┐│ + │ QUIC │ + │└ ─ ─ ─ ─ ┘│ + │┌ ─ ─ ─ ─ ┐│ + │ others │ + │└ ─ ─ ─ ─ ┘│ + └───────────┘ \ No newline at end of file diff --git a/protocol/network/wire.md b/protocol/network/wire.md deleted file mode 100644 index 9f4ad61b2..000000000 --- a/protocol/network/wire.md +++ /dev/null @@ -1,105 +0,0 @@ - -# IPFS Protocol on the Wire - -Authors: [Juan Benet](http://github.com/jbenet) - -Reviewers: - - -* * * - -This [spec](../../) describes the IPFS protocol on the wire. - -As explained in the [network spec](./), IPFS is [transport agnostic](./#transport-agnostic). We have the **hard constraint** of making IPFS work across _any_ duplex stream (an outgoing and an incoming stream pair, any arbitrary connection) and work on _any_ platform. - -To make this work, IPFS has to solve a few problems: - -- [Protocol Multiplexing](#protocol-multiplexing) - running multiple protocols over the same stream - - [multistream](#multistream) - self-describing protocol streams - - [multistream-select](#multistream-select) - a self-describing protocol selector - - [Stream Multiplexing](#stream-multiplexing) - running many independent streams over the same wire. -- [Portable Encodings](#portable-encodings) - using portable serialization formats -- [Secure Communications](#secure-communication) - using ciphersuites to establish security and privacy (like TLS). - -## Protocol-Multiplexing - -Protocol Multiplexing means running multiple different protocols over the same stream. This could happen sequentially (one after the other), or concurrently (at the same time, with their messages interleaved). We achieve protocol multiplexing using three pieces: - -- [multistream](#multistream) - self-describing protocol streams -- [multistream-select](#multistream-select) - a self-describing protocol selector -- [Stream Multiplexing](#stream-multiplexing) - running many independent streams over the same wire. - -### multistream - self-describing protocol stream - -[multistream](https://github.com/jbenet/multistream) is a self-describing protocol stream format. It is extremely simple. Its goal is to define a way to add headers to protocols that describe the protocol itself. It is sort of like adding versions to a protocol, but being extremely explicit. - -For example: - -``` -/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 - - -... -``` - -### multistream-selector - self-describing protocol stream selector - -[multistream-select](https://github.com/jbenet/multistream/tree/master/multistream-select) is a simple [multistream](https://github.com/jbenet/multistream) protocol that allows listing and selecting other protocols. This means that Protomux has a list of registered protocols, listens for one, and then _nests_ (or upgrades) the connection to speak the registered protocol. This takes direct advantage of multistream: it enables interleaving multiple protocols, as well as inspecting what protocols might be spoken by the remote endpoint. - -For example: - -``` -/ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 -/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 - - -... -``` - -### Stream Multiplexing - -Stream Multiplexing is the process of multiplexing (or combining) many different streams into a single one. This is a complicated subject because it enables protocols to run concurrently over the same wire. And all sorts of notions regarding fairness, flow control, head-of-line blocking, etc. start affecting the protocols. In practice, stream multiplexing is well understood and there are many stream multiplexing protocols. To name a few: - -- HTTP/2 -- SPDY -- QUIC -- SSH - -IPFS nodes are free to support whatever stream multiplexors they wish, on top of the default one. The default one is there to enable even the simplest of nodes to speak multiple protocols at once. The default multiplexor will be HTTP/2 (or maybe QUIC?), but implementations for it are sparse, so we are beginning with SPDY. We simply select which protocol to use with a multistream header. - -For example: - -``` -/ipfs/QmdRKVhvzyATs3L6dosSb6w8hKuqfZK2SyPVqcYJ5VLYa2/multistream-select/0.3.0 -/ipfs/Qmb4d8ZLuqnnVptqTxwqt3aFqgPYruAbfeksvRV1Ds8Gri/spdy/3 - -/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3 - - - -/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-bitswap/0.3.0 - - - - - - - - - - - - -... -``` - -## Portable Encodings - -In order to be ubiquitous, we _must_ use hyper-portable format encodings, those that are easy to use in various other platforms. Ideally these encodings are well-tested in the wild, and widely used. There may be cases where multiple encodings have to be supported (and hence we may need a [multicodec](https://github.com/jbenet/multicodec) self-describing encoding), but this has so far not been needed. - -For now, we use [protobuf](https://github.com/google/protobuf) for all protocol messages exclusively, but other good candidates are [capnp](https://capnproto.org), [bson](http://bsonspec.org/), [ubjson](http://ubjson.org/). - - -## Secure Communications - -The wire protocol is -- of course -- wrapped with encryption. We use cyphersuites similar to TLS. This is explained further in the [network spec](./#encryption). diff --git a/protocol/routing/DHT.md b/protocol/routing/DHT.md new file mode 100644 index 000000000..0f9ef8935 --- /dev/null +++ b/protocol/routing/DHT.md @@ -0,0 +1,28 @@ +IPFS DHT Protocol Spec +====================== + +Authors: David Dias + +Reviewers: + +TODOS: + +----------------------- + +> + +## Supports + +- Maintaining partial state of the network + - DHT + - kbucket + +## Overview + + + +explain: +- dht/coral, how the algo works +- kbucket +- each time a contact is made with a new peer, we check to see if it is a better candidate for our kbucket +- xor metric diff --git a/protocol/routing/README.md b/protocol/routing/README.md new file mode 100644 index 000000000..e4c0a1e12 --- /dev/null +++ b/protocol/routing/README.md @@ -0,0 +1,74 @@ +IPFS Routing Protocol Spec +========================== + +Authors: David Dias + +Reviewers: + +TODOS: + +----------------------- + +> This spec defines the routing protocol spec. Routing offers an interface for the features exposed by `Peer discovery` and `DHT`. The spec is a **Work In Progress**. + +## Supports + +- Routing primitives + - Publish and fetch content (also providing) +- Maintaining partial state of the network + - DHT + - kbucket + +## Overview + +The Routing Protocol is composed by three componenets, these are: +- Interface: Our routing primitives that are offered for the user, such as finding and publishing content, including the storage and lookup of metadata (Providers). +- Peer Discovery: Responsible for filling our kbucket with best candidates. +- Peer-to-peer Structured Overlay Network (DHT): Algorithm for the implicit network organization, based on [Coral](http://iptps03.cs.berkeley.edu/final-papers/coral.pdf) and [mainlineDHT](http://www.bittorrent.org/beps/bep_0005.html) + +``` +┌──────────────┐ +│ routing │ +└──────────────┘ +┌─────────┐┌───┐ +│discovery││DHT│ +└─────────┘└───┘ +``` + +In order for routing to work, we first have to pass the bootstrap state. Bootstrapping happens by connecting to a predefined "railing" peers list, shipped with the go-ipfs release and/or by discovery through mDNS. Once at least one peer is found and added to the kbucket, the routing changes to an active state and our peer becomes able to route and receive messages. + +## Routing + +For impl reference, check: https://github.com/ipfs/go-ipfs/blob/master/routing/routing.go#L19-L49 + +### Find a peer + +Finding a peer happens through an iterative process. We query the best candidate in our kbucket for the awarness of the peer we are looking for, if that peer isn't the ideal candidate, it will return 3 possible candidates from his kbucket and it will query those to check for the best candidate. We repeat the process until we find that peer + +_When searching for a peer, do we fetch the kbucket from a peer and see which peer we want to ping next or do we ask for a given Id to a peer and that peer replies to us with the best candidate (or itself if it is the case)?_ + +### Ping + +Ping mechanism (for heartbeats). Ping a peer and log the time it took to answer. + +_what if the Id doesn't exist? Is there any rule for non existing peers? Should we log time for best matches as well?_ + +### Provide + +Providing is the process of storing/updating the metadata (pointers) of where the blocks of a given file are stored/available in the IPFS network. What this means is that the DHT is not used for block discovery, but for the metadata which identifies where they are, instead. +When a node advertises a block available for download, IPFS stores a record in the DHT with its own Peer.ID. This is termed "providing". the node becomes a "provider". Requesters who wish to retrieve the content, query the DHT (or DSHT) and need only to retrieve a subset of providers, not all of them. (this works better with huge DHTs, and latency-aware DHTs like coral). + +We provide once per block, because every block (even sub-blocks) are independently addressable by their hash. (yes, this is expensive, but we can mitigate the cost with better DHT + record designs, bloom filters, and more) + +There is an optimistic optimization -- which is that if a node is storing a node that is the parent (root/ancestor) of other nodes, then it is much more likely to also be storing the children. So when a requester attempts to pull down a large dag, it first queries the DHT for providers of the root. Once the requester finds some and connects directly to retrieve the blocks, bitswap will optimistically send them the "wantlist", which will usually obviate any more dht queries for that dag. we haven't measured this to be true yet -- we need to -- but in practice it seems to work quite well, else we wouldnt see as quick download speeds. (one way to look at it, is "per-dag swarms that overlap", but it's not a fully correct statement as having a root doesn't necessarily mean a node has any or all children.) + +Providing a block happens as it gets added. Reproviding happens periodically, currently 0.5 * dht record timeout ~= 12 hours. + +### Get value + + + +### Put value + +_not 100% about this happens exactly. From what I understand, the IPFS node that is adding the file, breaks the file into blocks, creates the hashes and provides each single one of them. When do we execute a Put? Replicas are done through "Get", right?_ + diff --git a/protocol/routing/discovery.md b/protocol/routing/discovery.md new file mode 100644 index 000000000..88fac3897 --- /dev/null +++ b/protocol/routing/discovery.md @@ -0,0 +1,41 @@ +IPFS Peer Discovery Protocol Spec +================================= + +Authors: David Dias + +Reviewers: + +TODOS: + +----------------------- + +> + +## Supports + +- Peer discovery through + - mdns + - custom peers list + - random walking on the network + +## Overview + + +### bootstrap peer list + +List with known and trusted peers shipped with IPFS. + +- _How is this list updated?_ +- _Is this list updated periodically_? + +### random walk + +IPFS issues random Peer lookups periodically to refresh our kbucket if needed. For impl reference, see: https://github.com/ipfs/go-ipfs/blob/master/routing/dht/dht_bootstrap.go#L88-L109. + +### mDNS + +In addition to known peers and random lookups, IPFS also performs Peer Discovery through mDNS ([MultiCast DNS](https://tools.ietf.org/html/rfc6762)) + +-_How offen do we issue this searches?_ + + diff --git a/protocol/routing/figs/routing-overview.monopic b/protocol/routing/figs/routing-overview.monopic new file mode 100644 index 000000000..631136ca6 Binary files /dev/null and b/protocol/routing/figs/routing-overview.monopic differ diff --git a/protocol/routing/figs/routing-overview.txt b/protocol/routing/figs/routing-overview.txt new file mode 100644 index 000000000..e9552a8cd --- /dev/null +++ b/protocol/routing/figs/routing-overview.txt @@ -0,0 +1,6 @@ +┌──────────────┐ +│ routing │ +└──────────────┘ +┌─────────┐┌───┐ +│discovery││DHT│ +└─────────┘└───┘ \ No newline at end of file