Experiment with IDONTWANT on Publish()

[raulk]

The IDONTWANT control signal was recently introduced to instruct peers to suppress the propagation of messages that a node has already received from other sources (or that it is otherwise aware of). This mechanism aims to reduce unnecessary message duplication within the network.

Currently, go-libp2p-pubsub emits ⁠IDONTWANT signals only while processing incoming messages. These signals are dispatched early in the inbound pipeline (specifically, during RPC prevalidation and before messages are handed off to the application). Tests performed in controlled environments by various groups have confirmed measurable reductions in redundant traffic from this new feature.

However, there is an additional opportunity to further decrease superfluous network traffic: sending an ⁠IDONTWANT signal to mesh peers when the app invokes ⁠Publish. In this scenario, the ⁠IDONTWANT would precede the corresponding Publish RPC (in fact, the Publish RPC itself may even be discarded if the peer had previously signalled disinterest via an ⁠IDONTWANT).

This optimization is particularly useful in highly synchronized environments, such as the Ethereum network. When a block is produced, CL nodes typically race to source blobs locally (from the EL), construct sidecars, and disseminate them to peers. In 60–70% of cases the required data is already present in the local EL. In these situations, the application issues a ⁠Publish and does not need to receive the same message from its peers, so triggering an IDONTWANT would correctly signal that.

[raulk]

Apparently rust-libp2p already behaves this way (cc @jxs) 🚀

[jxs]

Hi Raúl. Rust-libp2p allows has implemented sending IDONTWANT messages when publishing a message: libp2p/rust-libp2p#5773.
But now that I re-read your description, it has a small difference from it, in rust-libp2p the IDONTWANT is sent after the message to be published, we can easily send it before the published message, it probably makes more sense.
cc @AgeManning

[MarcoPolo]

This makes sense. I'm wondering if we could do a bit better.

The problem we're trying to solve here is to reduce duplicates in the case that a large percentage (most?) of the network already has some message that will be propagated on a topic.

In this case, you want to avoid eagerly pushing the message to your peers that likely already have it. Maybe by first sending an IDONTWANT, add the message to our local message cache (so it gets gossiped), and delay pushing the message by a small amount (rtt? rtt/2?) to allow neighbors to give us their IDONTWANT.

We likely need a new API for this case because we need the application to tell us this message is likely on the network already.

That said, I'm not opposed to sending an IDONTWANT in front of the message as a first and easy step.

[AgeManning]

Yep. I don't see a problem with preceding a publish with an IDONTWANT. We send it directly after, so, in principle the scenario we are saving is the case that the message can propagate two hops by the time we finish sending a single publish (i.e we send to peer 1, and it sends it to peer 2 which then sends it back to us, by the time we finish sending to peer 2 ourselves).

If I understand @MarcoPolo's comment, I think the only different in logic that we already have is artificially adding a delay in forwarding. We can do this currently without any code changes via artificially delaying the validation. We need to complete validation before we forward a message, this already has some inherit delay, but we could artificially increase it. I think doing this is playing the trade-off game between message propagation latency and bandwidth reduction.

The message latency is a big concern in the block topic, we get messages arriving late and it causes consensus issues currently.

[MarcoPolo]

in principle the scenario we are saving is the case that the message can propagate two hops by the time we finish sending a single publish (i.e we send to peer 1, and it sends it to peer 2 which then sends it back to us, by the time we finish sending to peer 2 ourselves)

I think the case @raulk has in mind is that two validators have the same message (from the EL) and they are going to publish it at the same time. For example, say they have all blobs required to make a column, and they both publish the column at roughly the same time.

The "boomerang" case that you can send a message out and get the message back to you from another peer before you send it to them is a good one that I hadn't considered. It would also be fixed by this.

We can do this currently without any code changes via artificially delaying the validation.

Ah right. This delay might be enough since we can send the IDONTWANT prior to validation.

The message latency is a big concern in the block topic, we get messages arriving late and it causes consensus issues currently.

👍 That's why the application needs to be the one deciding if a message is safe to delay or not. The delay helps the case where two or more peers publish the same thing at roughly the same time, it hurts all other cases.

[Nashatyrev]

Done in JVM libp2p: libp2p/jvm-libp2p#386