eth/filters, core/filtermaps: safe chain view update

[zsfelfoldi]

This PR changes the chain view update mechanism of the log filter. Previously the head updates were all wired through the indexer, even in unindexed mode. This was both a bit weird and also unsafe as the indexer's chain view was updates asynchronously with some delay, making some log related tests flaky. Also, the reorg safety of the indexed search was integrated with unindexed search in a weird way, relying on syncRange.ValidBlocks in the unindexed case too, with a special condition added to only consider the head of the valid range but not the tail in the unindexed case.

In this PR the current chain view is directly accessible through the filter backend and unindexed search is also chain view based, making it inherently safe. The matcher sync mechanism is now only used for indexed search as originally intended, removing a few ugly special conditions.

The PR is currently based on top of #31642
Together they fix #31518 and replace #31542

[rjl493456442]

My general feeling is that the chain view management was
implemented incorrectly from the beginning.

Whenever a potential reorg occurs, the current implementation
updates the new chain view in the FilterMaps without explicitly
notifying components like (a) the indexer and (b) the matcher
synchronously.

Instead, the new chain view is updated asynchronously, and
any potentially invalid indexed data or matched results are
removed afterward.

This approach may introduce a number of issues while offering
little to no real benefit.

Ideally, it should be implemented that: once reorg occurs, immediately
terminate the ongoing indexer/matcher if they are affected, discarding
the affected data and resume.

[zsfelfoldi]

My general feeling is that the chain view management was implemented incorrectly from the beginning.

Whenever a potential reorg occurs, the current implementation updates the new chain view in the FilterMaps without explicitly notifying components like (a) the indexer and (b) the matcher synchronously.

Blocking SetTarget would have its own dangers. It's always a bit tricky with any kind of extra chain processing as even blocking a chain subscription can disrupt the entire client operation.
The indexer does pretty much what you expect though, it checks for target updates on the channel frequently and discards the map rendering immediately if the iterator is already past the point where the new target forks off from the previous one.
The matching could be done differently for sure but the thing is that for efficiency reasons the entire matching process is epoch based and one epoch contains tens of thousands of blocks so canceling the processing of the latest epoch just because the last 1-2 blocks have been reorged might not be the most efficient.

[rjl493456442]

Can I just confirm that

s.syncRange.ValidBlocks is always consistent with the result returned by s.filter.indexedLogs right?

[rjl493456442]

hmm, probably not.

e.g., the indexed view was A->B, then A->B->C->D; then it's changed to A->B'->C'->D';

The match backend was initialized with at A->B; the s.syncRange.ValidBlocks returned is A (B was removed due to indexed view reorg). s.syncRange.ValidBlocks is aligned with the indexed view.

However, the indexed view might be forked with the current block chain (e.g. chain is rewound to A'->B''->C''->D'') and this view hasn't be propagated into the indexer yet.
In this case, the s.syncRange.ValidBlocks will be trimmed by the intersection of indexed view and chain view.

This subset should be totally trusted and canonical.

@zsfelfoldi is it correct?

[zsfelfoldi]

Yes, it's correct. The block number range in syncRange.ValidBlocks always refers to syncRange.IndexedView which is usually the same as the local reference view s.chainView but has this small delay because of async indexing so in case they are different the guaranteed correct range is trimmed further so that s.matchRange and s.matches always stay consistent with s.chainView.
The indexer's event handler will check for new heads before the new matcher sync iteration so in the next sync the two views will hopefully match and the results of the next search iteration will all be considered safe and the search can be successfully finished (unless of course there is a next reorg happening very quickly).

[rjl493456442]

I’ve tried my best to review this PR, and it looks good to me.

However, my understanding of this package isn’t very comprehensive, so I can’t guarantee that the new filtermap implementation is completely issue free.

But the flaky test appears to be resolved, which is a good sign.