Gossipsub: Partial Messages Extensions

[MarcoPolo]

In draft, will fill this body when ready.

[cskiraly]

As mentioned in our call and prior discussions, this is what we've described in the past as bitmap-based IHAVE/IWANT.
You can find prior art on this e.g. here: https://ethresear.ch/t/fulldas-towards-massive-scalability-with-32mb-blocks-and-beyond/19529#possible-extensions-13

We have already discussed this a few times as the way forward, but haven't arrived to spec it. Hence, I have also started preparing a draft spec on the same a few weeks ago. You can find my draft here:
https://notes.ethereum.org/jAaSLjGBT92iyiNMoSyKNg

It is a bit unfinished, hence I didn't push it here on the repo yet :-) In general I think it is quite aligned with what you propose here. Let's see how we can merge our views into one generic spec.

As I say in my draft above, a structured message ID is used to identify a segment of a larger message. In another view, we can see these as messages in a larger batch of messages. One can easily see that the two are almost exactly the same.

[cskiraly]

Partial message validation is a crucial aspect to this, in fact we can do fast forwarding in FullDAS because we can validate every single message segment individually.

I would dedicate a section to this in the spec (trying to write it up).

[cskiraly]

@MarcoPolo proposed improvements to the partial message spec here
https://github.com/cskiraly/libp2p-specs/tree/csaba/partial-message

In my version I had both partialIHAVE and partialIWANT.

I think I see your point in focusing on the IWANT part, as it might allow us to delegate all the logic to the upper layer, similar to how you can do partial requests in other protocols where you need pieces only (e.g. HTTP Accept-Ranges for resume, and similar use cases). We still need the other node to send the IHAVE (the normal, full IHAVE, and we still would need the logic in our node not to fetch the whole message, so either a hook before IWANT or some call to say "for this I already have something, please send a PartialIWANT with this extra metadata".

If instead we also define the Partial IHAVE, we can do more. In the FullDAS implementation I'm doing these partial IHAVEs to some extent already, because I'm segmenting the column to cell-level messages. We can also give a bitmap encoding to these, as it was proposed there.

Regarding keeping the metadata opaque, I think it is an interesting idea, but it seems to me it requires a more complex API. At the end of the day we do need a protocol layer that is aware of the underlying segment structure. We can try to have this layering of the structure aware and structure unaware part, but I feel this will lead to inefficiencies at some point.

[MarcoPolo]

Let's see how we can merge our views into one generic spec.

That sounds great to me!

Partial message validation is a crucial aspect to this, in fact we can do fast forwarding in FullDAS because we can validate every single message segment individually.

Agreed.

The other use case is faster verification of complete messages. Imagine the fusaka das use case where we are only missing a single cell, we don't need to verify the full column again, just the missing cell.

On fast forwarding, there's a couple different strategies we could do here, but one that stands out to me follows this simple rule:

• Only eagerly push/forward cells we did not already have locally.

The underlying idea is that, as the mempool works today, if you have a cell locally odds are high your peer has it as well (again from your very helpful post: https://ethresear.ch/t/is-data-available-in-the-el-mempool/22329). As we change the mempool, we'll change this strategy as well.

We still need the other node to send the IHAVE (the normal, full IHAVE, and we still would need the logic in our node not to fetch the whole message, so either a hook before IWANT or some call to say "for this I already have something, please send a PartialIWANT with this extra metadata".

This is hard because we don't have a way to link the GroupID to the full message ID present in the IHAVE. One approach is to go all in with partialIHAVEs and use that tell a peer that we have all of the parts of this group, and never send an IHAVE if a peer supports partial message extensions.

Regarding keeping the metadata opaque, I think it is an interesting idea, but it seems to me it requires a more complex API.

I think keeping the metadata opaque and completely application defined is the correct approach. It lets the application define the optimal encoding of the missing. For the das use case bitmaps are very good, but I could also imagine a use case where ranges are desired, or maybe even something like a rateless IBLT.

I feel this will lead to inefficiencies at some point.

What inefficiencies do you have in mind here?