Recently, I discovered an interesting database project called OrbitDB. I want to create a decentralized public anime metadata database using OrbitDB's decentralized implementation through libp2p. However, understanding and configuring the libp2p-related content has been the most difficult part of the development process. Fortunately, the official documentation has released a high-quality beginner's tutorial in the past two weeks, which provides a lot of relevant knowledge. It's worth summarizing the content in a blog post.
WebRTC with js-libp2p
Why Choose WebRTC#
What is WebRTC#
WebRTC, short for Web Real-Time Communication, is an API that supports real-time voice or video conversations in web browsers. With WebRTC, browsers can establish peer-to-peer connections, allowing direct data transfer between two browsers. WebRTC protocol is supported by mainstream browsers, and when combined with js-libp2p, it allows browsers to act as libp2p nodes in the libp2p network.
JS-libp2p Connectivity and Capability#
There are many language implementations of libp2p, but each implementation has different levels of completeness and limitations. The official website provides the implementation progress and limitations of each language.
libp2p implementation progress
Due to browser limitations, browsers cannot directly transmit data using TCP and QUIC protocols. Currently, there are only three protocols available for creating independent libp2p nodes in browsers: WebSocket, WebTransport, and WebRTC. The official website also provides the connectivity results between different libp2p protocols and explains why browsers do not support TCP and QUIC protocols.
Connectivity between libp2p protocols
According to the connectivity results provided by the official website, currently WebSocket and WebTransport can only connect to servers and do not support direct browser-to-browser connections. Among all the implementation solutions, only js-libp2p combined with WebRTC can treat browsers as independent nodes.
Limitations and Solutions of WebRTC#
The most important issue when establishing a P2P connection is how to make two browsers discover each other so that the nodes can establish subsequent connections. Two solutions are provided: STUN servers and TURN servers.
- STUN servers: Due to the presence of many NAT structures in the network, it is difficult for browsers to discover each other directly through public IP addresses. STUN servers help browsers discover each other's nodes, facilitating the establishment of connections. There are many free public STUN servers available.
Free public STUN servers - TURN (Relay) servers: If direct P2P connections cannot be established between browsers, even if STUN services help nodes discover each other, TURN servers can be used to relay all traffic of P2P connections. However, using TURN servers has higher costs and is not an ideal choice.
When browsers establish P2P connections, they may need to send some metadata or signaling information. The WebRTC standard does not specify these contents. Developers need to implement their own signaling exchange solutions.
In libp2p, a new solution is designed to solve the above two problems. It is solved through a libp2p relay node. The libp2p relay node plays two roles in the network:
- Circuit Relay V2: When browser nodes cannot establish P2P connections directly, using TURN services is costly. Circuit Relay V2 can solve this problem well because it is decentralized. It allows active nodes in the network to help browsers discover each other and relay the required traffic when P2P connections cannot be established.
- PubSub Peer Discovery: The above solution solves the issue of whether nodes can communicate with each other, while PubSub Peer Discovery solves the problem of automatic connection between nodes. GossipSub mode is an implementation solution of PubSub Peer Discovery. Nodes subscribe to a topic, and whenever a new node joins and subscribes to the same topic, PubSub Peer Discovery broadcasts the new node's address to all nodes subscribed to that topic. Each node will try to connect with the new node after receiving the address. However, it is important to note that broadcasting in a large network consumes a lot of resources, and improper use may cause congestion and loss of reliability, so this solution may not be suitable for production environments.
Connection Process#
Regarding how two nodes establish a P2P connection and the role of the libp2p relay node in the connection, the official documentation provides a flowchart that roughly describes the workflow.
In terms of this process, the general idea is that when two nodes establish a connection, they need to know the address of the Relay node first. They initiate a WebSocket connection with the Relay node. After both nodes successfully establish a stable connection with the Relay node and subscribe to the same GossipSub topic, the Relay node will inform the nodes subscribed to the same topic about each other's multi-addresses (multiaddr) through Circuit Relay V2. A multi-address is in the following format:
// Multi-address of a node
// Using WebSocket on port 9001 with WebRTC support
/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/webrtc/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
// Not using WebSocket on port 9002 with WebRTC support
/ip4/127.0.0.1/tcp/9002/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/webrtc/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
// Using WebSocket on port 9001 without WebRTC support
/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
// Not using WebSocket on port 9002 without WebRTC support
/ip4/127.0.0.1/tcp/9002/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
Finally, the two nodes will attempt to establish a WebRTC connection using the multi-addresses, completing the P2P connection.
The official documentation also provides a more detailed flowchart of the WebRTC connection process.
After discovering each other through the libp2p relay node, the WebRTC connection can be established. First, each node obtains its public IP and port from the STUN server. Then, the initiating node creates an RTCPeerConnection object (used to manage the WebRTC connection state) and creates a DataChannel and SDP (Session Description Protocol, used to describe session information) to prepare for the connection. The initiating node creates a signaling (webrtc-signaling) defined in libp2p and sends the SDP to the target node through the libp2p relay node. When the target node receives the SDP, it also creates an RTCPeerConnection and SDP to reply to the sender. The sender receives the SDP and records it. At this point, the two nodes complete the handshake and exchange SDP information.
Then, the ICE (Interactive Connectivity Establishment, used to find the best connection path in P2P connections) adds the other node to the connection management object RTCPeerConnection. Finally, the connection is encrypted using Noise (a communication protocol), completing the connection.
This concludes the theoretical article. To gain a deeper understanding of the content, you can continue reading the practical article:
Creating a libp2p Network and Establishing Connections Using WebRTC with JS (Practice)