Rust Libp2p Versions Save

See individual changelogs for details.

See individual changelogs for details.

The most ergonomic version of rust-libp2p yet!

We've been busy again, with over 250 PRs being merged into master since v0.52.0 (excluding dependency updates).

Backwards-compatible features

Numerous improvements landed as patch releases since the v0.52.0 release, for example a new, type-safe SwarmBuilder that also encompasses the most common transport protocols:

let mut swarm = libp2p::SwarmBuilder::with_new_identity()
    .with_tokio()
    .with_tcp(
        tcp::Config::default().port_reuse(true).nodelay(true),
        noise::Config::new,
        yamux::Config::default,
    )?
    .with_quic()
    .with_dns()?
    .with_relay_client(noise::Config::new, yamux::Config::default)?
    .with_behaviour(|keypair, relay_client| Behaviour {
        relay_client,
        ping: ping::Behaviour::default(),
        dcutr: dcutr::Behaviour::new(keypair.public().to_peer_id()),
    })?
    .build();

The new builder makes heavy use of the type-system to guide you towards a correct composition of all transports. For example, it is important to compose the DNS transport as a wrapper around all other transports but before the relay transport. Luckily, you no longer need to worry about these details as the builder takes care of that for you! Have a look yourself if you dare here but be warned, the internals are a bit wild :)

Some more features that we were able to ship in v0.52.X patch-releases include:

• stable QUIC implementation
• for rust-libp2p compiled to WASM running in the browser
  • WebTransport support
  • WebRTC support
• UPnP implementation to automatically configure port-forwarding with ones gateway
• option to limit connections based on available memory

We always try to ship as many features as possible in a backwards-compatible way to get them to you faster. Often times, these come with deprecations to give you a heads-up about what will change in a future version. We advise updating to each intermediate version rather than skipping directly to the most recent one, to avoid missing any crucial deprecation warnings. We highly recommend you stay up-to-date with the latest version to make upgrades as smooth as possible.

Some improvments we unfortunately cannot ship in a way that Rust considers a non-breaking change but with every release, we attempt to smoothen the way for future upgrades.

#[non_exhaustive] on key enums

We've identified that adding a new "message" to the ToSwarm enum is a common cause for breaking changes. This enum is used by plugins (i.e. NetworkBehaviours) to communicate with the Swarm. Similarly, the FromSwarm enum is used to inform plugins about state changes. By adding #[non_exhaustive] to these and other enums we enable future additions to be non-breaking changes.

For example, we plan to allow NetworkBehaviours to share addresses of peers they've discovered with each other. Previously, we had to queue this feature until the next breaking change whereas now, we can simply ship it as soon as it is ready!

Thanks to @dhuseby for getting the ball rolling on this one. See PR 4581 for details.

Smarter keep-alive algorithm

Prior to this release, a protocol (i.e. ConnectionHandler) had to diligently track streams and other state to correctly compute the KeepAlive for the connection. A bug in this computation sometimes caused the connection to be shut down, despite it still being in use by the protocol. Earlier this year, I thought "there is gotta be a better way" and drafted up some ideas. Eventually @mxinden and myself hashed out the key-points around being smart in regards to keeping connections alive:

• By default, any existing stream will keep the connection alive.
• Protocols like ping can opt out of this with Stream::ignore_for_keep_alive().
• ConnectionHandler::connection_keep_alive only comes into effect once a connection would be shut down. This allows protocols like Circuit Relay V2 to keep the connection alive despite there not being any streams.

The nice benefit for you as a user is that connections are shut down more aggressivley, minimizing system resource use. Should you want to keep idle connections alive nonetheless, you can configure an additional delay using libp2p::swarm::Config::with_idle_connection_timeout.

This also makes wiritng your own protocols simpler.

The tracking issue for this effort was #4306 if you wanted to have a closer look. Special thanks to @leonzchang for landing many small PRs that made this possible!

Be gone ConnectionHandler::Error!

Talking about closing of connections, say goodbye to ConnectionHandler::Error and its related components. It took many months of careful planning, numerous discussions, failed attempts and s-e-v-e-r-a-l refactorings but we finally delete ConnectionHandler::Error with this release.

There's lots to be excited about with this change; let me explain. Deleting ConnectionHandler::Error also means:

• Removing a type-parameter from SwarmEvent. No more

    SwarmEvent<MyBehaviourEvent, Either<Either<Either<Either<Either<io::Error, Void>, io::Error>, io::Error>, Void>, io::Error>, StreamUpgradeError<Void>>, Void>
  

  Just:

  SwarmEvent<MyBehaviourEvent>
  

  Isn't that neat? :tada:
• ConnectionHandlerEvent::Close is gone too and together with it, a type parameter on its definition!
• FromSwarm also as a type-parameter removed which makes it easier and more concise to reference!
• Fewer force-closing of connections. Prior to this change, a ConnectionHandler could simply close an entire connection at any point, despite the connection still being in use by other protocols. (Remember that protocols are multiplexed over a single connection.) For example, an error while making a reservation with a relay would lead to the entire relayed connection being closed, even though other protocols might still have been using it.

This means more reliable connections and protocol executions as no one is pulling the rug out from underneath your protocol's legs as they run across the wire :running:

tracing

Protocols don't actually have legs, Thomas.

That is true. But if we imagine for a moment that they do, using the latest migration to tracing we can now watch them!

I am not sure if @eserilev knew what they were signing up for when they suggested to tackle #1533 back in August. A lot of impressive work and pretty much exactly three months later though, we merged #4282 which:

• migrates the entire workspace from log to tracing
• adds several spans to interesting functions
• showcases how to ingest this data into Jaeger

Enough talking, here is a screenshot:

Here we can see what one particular call to Connection::poll actually did. We can see how much time is spent in the respective identify and ping handlers as well as interaction with the stream multiplexer.

We put some consideration in to which spans might be useful but we are excited to hear from users how useful they are and what else might need instrumentation.

PollParameters has been removed

A relict from earlier designs of rust-libp2p is finally gone too: PollParameters. The NetworkBehaviour::poll function now just takes a &mut Context as you'd expect it to, without any additional clutter. All features of PollParameters have previously already been deprecated and alternatives were offered in terms of events.

If you enforce no warnings in your codebase, you should be able to just delete that (unused) parameter from the function.

Credits

This release would not have been possible without the contributions of many people. Thank you to (in alphabetical order):

• @0x7CFE
• @0xcrust
• @ackintosh
• @AgeManning
• @arpankapoor
• @arsenron
• @bgins
• @binsta
• @b-zee
• @chirag-bgh
• @clebrin
• @contrun
• @dariusc93
• @Dejiah
• @dgarus
• @dhuseby
• @divagant-martian
• @DougAnderson444
• @dvorakchen
• @eserilev
• @gurinderu
• @hanabi1224
• @joshuef
• @kayabaNerve
• @kpp
• @leonzchang
• @maqi
• @Marcel-G
• @mcamou
• @melekes
• @monoid
• @MOZGIII
• @nathanielc
• @oblique
• @onur-ozkan
• @p0mvn
• @PanGan21
• @pradt2
• @p-shahi
• @quininer
• @radiohertz
• @randommm
• @shamil-gadelshin
• @startup-dreamer
• @StemCll
• @sufferpurrityxd
• @tcoratger
• @thomas-senechal
• @umgefahren
• @whtsht
• @yerayrodriguez
• @zeeshanlakhani
• @zrus
• @zvolin

Closing

As always, a detailed log of changes for every release can be found in our repository: CHANGELOG.md. If you are struggling with an upgrade, feel free to tag Max (@mxinden) or myself (@thomaseizinger) in a PR and we'll try to help.

Happy coding!

See individual changelogs for details.

See individual changelogs for details.

See individual changelogs for details.

See individual changelogs for details.

Automatic kademlia client/server mode

Let's get the biggest one out the way first, I promise the other points are easier explained but equally exciting. The tl;dr is: Healthier Kademlia routing tables and an improved developer experience.

If you don't know about Kademlia's client/server mode, checkout the specs.

With the v0.52 release, rust-libp2p automatically configures Kademlia in client or server mode depending on our external addresses. If we have a confirmed, external address, we will operate in server-mode, otherwise client-mode. This is entirely configuration-free (yay!) although follow-up work is under-way to allow setting this manually in certain situations: #4074.

We can now do the following:

1. As soon as we learn about an external address (e.g. via AutoNAT), we activate server mode of Kademlia.
2. Activating server-mode means we allow inbound requests, this is a change in our set of supported protocols.
3. The change is detected automatically and reported to all protocols as ConnectionEvent::LocalProtocolsChange.
4. libp2p-identify picks up this change and pushes it to all connected remote nodes.
5. Remote nodes can instantly put us into their routing table.

To implement this, several other features/issues had to be fixed. If you are interested in the details, read on:

• Simplify the scoring mechanism of external addresses: #3954

  As a consequence, the observed address reported by identify is no longer considered an external address but just an address candidate. Checkout the changelog-entry for a way of restoring the old behaviour.

• Changes to the supported protocols are now detected at runtime and communicated to all protocols: #3651.

  Previously, a protocol could retrieve the supported protocols via PollParameters::supported_protocols. This list however was computed at start-up and was static. Now, ConnectionEvent has two new variants:

  pub enum ConnectionEvent<'a> {
    // existing variants omitted ...
  
    /// The local [`ConnectionHandler`] added or removed support for one or more protocols.
    LocalProtocolsChange(ProtocolsChange<'a>),
    /// The remote [`ConnectionHandler`] now supports a different set of protocols.
    RemoteProtocolsChange(ProtocolsChange<'a>),
  }
  
  pub enum ProtocolsChange<'a> {
    Added(ProtocolsAdded<'a>),
    Removed(ProtocolsRemoved<'a>),
  }
  

  ProtocolsAdded and ProtocolsRemoved are iterators over a (new) type called StreamProtocol.

• Protocols are now enforced to be valid UTF-8 strings: #3746.

  This was always the case in the specs but the rust-libp2p implementation was lagging behind here and improperly represented them as bytes internally.

• Local changes to our protocols (i.e. a node going from Kademlia server to client mode) are now immediately pushed to the remote via the /ipfs/id/push/1.0.0 protocol: #3980.

Not only does this work out-of-the-box and thus improves the developer experience of rust-libp2p, it should also result in a much more useful and up-to-date routing table for all nodes on a Kademlia DHT.

Type-safe /p2p multiaddresses

The /p2p protocol of multiaddresses specifies the identity of a peer in the form of a PeerId such as 12D3KooWETLZBFBfkzvH3BQEtA1TJZPmjb4a18ss5TpwNU7DHDX6. Yet for the longest time, the type-definition of the /p2p protocol looked like this:

pub enum Protocol<'a> {
    // omitted other variants ...
    P2p(Multihash),
}

Every PeerId is a valid Multihash but not vice-versa. Dang! That is a lot of "impossible" errors that the type-system should avoid.

Thanks to a lot of work, it now looks like this:

pub enum Protocol<'a> {
    // omitted other variants ...
    P2p(PeerId),
}

More consistent event/command naming

Naming is hard and humans are creatures of habit. Thus, once familiar with certain names, it is often hard to see how they make absolutely no sense at all to newcomers.

In the v0.52 release we renamed several associated types and enums which hopefully make the message-passing system implemented in rust-libp2p easier to grasp.

A quick recap:

• A NetworkBehaviour represent a protocol's state across all peers and connections.
• A ConnectionHandler represents a protocol's state for a single connection to a peer.
• Multiple NetworkBehaviours are composed into a tree using #[derive(NetworkBehaviour)] and run inside a Swarm

NetworkBehaviours can emit events to the Swarm. This used to be called OutEvent. Now, its aptly named ToSwarm:

pub trait NetworkBehaviour {
    type ToSwarm;
  
    // functions and other types omitted ...
}

Returning one of these events was previously done with a NetworkBehaviourAction::GenerateEvent, a type-name so long you were grateful for autocomplete. These actions are essentially commands that are issued to the swarm. What could possibly be a good name for that? I present:

pub enum ToSwarm<TOutEvent, TInEvent> {
    GenerateEvent(TOutEvent),
    
    // other variants omitted ..
}

We followed the same strategy for ConnectionHandler. A ConnectionHandler can receive messages from a NetworkBehaviour via ToSwarm::NotifyHandler (gosh, that was so much easier to write, why didn't we do this earlier?) and send message to its NetworkBehaviour. The associated types defining these messages are now called ToBehaviour and FromBehaviour, representing where the message is going / coming from. Previously, they carried the generic names InEvent and OutEvent which had me utterly confused when I first started working on rust-libp2p.

To wrap it all up, the ConnectionHandlerEvent::Custom variant is now called ConnectionHandlerEvent::NotifyBehaviour, making it clear what happens to types returned here.

Improved ergonomics around stream errors

Oh, isn't this one of my favourites!

Ever wondered why ConnectionHandlerUpgrErr had what felt like 10 layers of nested enums? So did we and went ahead and fixed this in #3882.

This is what it looked like before:

pub enum ConnectionHandlerUpgrErr<TUpgrErr> {
    Timeout,
    Timer,
    Upgrade(UpgradeError<TUpgrErr>),
}

pub enum UpgradeError<E> {
  Select(NegotiationError),
  Apply(E),
}

pub enum NegotiationError {
  ProtocolError(ProtocolError),
  Failed,
}

pub enum ProtocolError {
  IoError(io::Error),
  InvalidMessage,
  InvalidProtocol,
  TooManyProtocols,
}

Now, it looks like this:

pub enum StreamUpgradeError<TUpgrErr> {
    Timeout,
    Apply(TUpgrErr),
    NegotiationFailed,
    Io(io::Error),
}

But that is not it!

Previously, we would give you one of these ConnectionHandlerUpgrErr when an inbound stream failed. But, what exactly does ProtocolError::InvalidMessage for example mean for an inbound stream? How would we even figure out, which protocol (read ConnectionHandler) this should be dispatched to if we failed to negotiate the protocols?

Well, you might have guessed it. It is impossible to dispatch this to the correct one, so we just informed all protocols. But that was pretty useless. If we don't know, which stream a protocol belongs to, we shouldn't just inform all of them. Thus, in #3605 we stopped this which removes several "impossible" error paths.

Simpler noise interface

Since its introduction, the libp2p-noise crate supported a wide range of handshake patterns. The libp2p specs however only documents the XX handshake: libp2p/specs/noise.

Supporting additional patterns introduced significant complexity to the codebase. We decided to deprecate and remove them. This significantly reduced the complexity of the libp2p-noise implementation (-1000 LoC!). Additionally, it allowed us to finally adopt the naming conventions we have been pursuing across the workspace for libp2p-noise.

Using the noise handshake is now as simple as:

let config = libp2p::noise::Config::new(&keypair).unwrap();

Request-response protocols using serde

A simple, yet impactful contribution was made by @dgarus in #3952:

• libp2p::request_response::cbor::Behaviour<Req, Res>
• libp2p::request_response::json::Behaviour<Req, Res>

These two, new type-aliases come with a pre-configured CBOR/JSON codec and only need to be parameterised by the Req and Res types. The only requirement is that the types implement serde::{Serialize,Deserialize}. Defining a request-response protocol has never been easier in rust-libp2p!

As with any code that serializes data structures, be aware that any changes to it can easily be breaking and thus not backwards-compatible with older versions.

Hole-punching for QUIC

Despite still being in alpha, our QUIC implementation can now establish direct connections across certain NATs, i.e. hole-punching! This was contributed by @arpankapoor in #3964.

You don't need to configure anything special besides the dcutr behaviour. See the dcutr-example in our repository for details.

Thanks!

A big thanks to all people that contributed to this release. In alphabetical order:

• @AgeManning
• @arpankapoor
• @b0xtch
• @b-zee
• @creativcoder
• @dariusc93
• @dgarus
• @DougAnderson444
• @drHuangMHT
• @elenaf9
• @felinira
• @galargh
• @hanabi1224
• @helloimalemur
• @jsantell
• @kckeiks
• @leviathanbeak
• @linoscope
• @melekes
• @nathanielc
• @obi1kenobi
• @oblique
• @ozkanonur
• @ozwaldorf
• @PopBogdan97
• @shamil-gadelshin
• @StemCll
• @stonecharioteer
• @stormshield-pj50
• @tcoratger
• @tthebst
• @umgefahren
• @uwejan
• @vnermolaev
• @wizeguyy
• @yellowred

Closing

Well done for making it to the end!

As always, a detailed log of changes for every release can be found in our repository: CHANGELOG.md. If you are struggling with an upgrade, feel free to tag Max (@mxinden) or myself (@thomaseizinger) in a PR and we'll try to help.

Happy coding!

See individual changelogs for details.

See individual changelogs for details.