This is about making it possible to play with thread events, but not necessarily storing them in the persisted event cache yet.

Core types

• a timeline event may have a ThreadSummary, if it's a thread root
  • see also Threads: attach a summary to thread roots #5036
  • count (likely not the raw event count, but renderable event count)
  • latest event id (observers can use the load_or_fetch_event() method to retrieve it from the cache when needed)
  • participated: TBD
  • maybe_outdated: bool indicating whether the summary might be outdated, after a gappy sync
• for each thread (identified by room id + root event id), a ThreadEventCache { linked_chunk: LinkedChunk<TimelineEvent, ThreadGap> } (ThreadGap could be the same as Gap)
• the RoomEventCache contains a mapping of all the threads roots to their ThreadEventCache

Tasks

• Reconcile or unify RoomEvents event chunks with AllEventsCache events #3886
• event cache: update storage to store the events' content in a separate table #4841
• Extract the bundled aggregation from a thread root, and feed it into the RoomEventCache, which forwards it to the ThreadEventCache ONLY if the thread was not known beforehand
  • Save the bundled thread's latest event into the event cache (don't insert into the thread linked chunk, only save it into the events table with save_event())
• Have a linked chunk per thread.
• Allow subscribing to a thread by receiving the initial events in that thread, if known, and a stream of VectorDiff<Event> (this will power the thread-focused timeline).
  • the timeline loads related events itself (to retrieve reactions and so on)
  • make sure relations are correctly ordered, namely edits
• implement back-pagination at the ThreadEventCache level; this will be used by the thread-focused timeline paginate_backwards() method.

How to behave with respect to sync and (main) back-pagination?

MVP

With this version, we might have many empty threads (every time there's a gappy sync), and thread counts might be incorrect.

• When a new sync event is received, and it belongs to a thread:
  • if it's a known thread: append the event to the thread linked chunk
  • if it belongs to an unknown thread: don't push the event, only record the thread summary for the thread root (latest event + increase count?)
• When a sync is gappy (limited):
  • clear all the thread linked chunks 😔 we don't know which threads have missing events, so we have to assume each could've been modified
• When a back-pagination result is returned:
  • we don't know where to insert the (main) back-paginated events inside the thread, so only update the thread summary for the thread root (latest event if it wasn't set + update count)

Smart Edition

• when a new sync event is received, and it belongs to a thread:
  • if it's a known thread: append the event to the thread
  • if it belongs to an unknown thread:
    • push an in-thread empty back-pagination token
    • push the event
• when a sync is gappy (limited):
  • for each thread,
    • if the thread had an in-thread empty back-pagination token (⚠ needs lookup in DB, or for the linked chunk to be fully loaded in memory)
      • remove it
      • remove all events after it
    • push an empty back-pagination token
    • (push the related in-thread events)

The trick is that the empty back-pagination token will make it so that we resume back-pagination from the end of the thread to the start: duplicated events will be removed and reinserted in their rightful position. Gappy syncs may happen, but only have a limited impact on the thread as a whole (until the previous back-pagination).

Big Brain Edition: "waved paginations"

This is more or less an intuition of what we could do to not have to implement a catchup mechanism server-side, for threads.

A thread linked chunk's gap may be one of two variants:

• either a thread backpagination token,
• or a main (non-threaded) backpagination token

When a sync is gappy: push the main backpagination token to every single thread.

When a main backpagination is finished: for each thread, find the main backpagination token, if it was there, and insert in-thread events in their rightful positions.

When an in-thread backpagination happens, ignore main back-pagination tokens, and only use in-thread pagination tokens. Unclear it's sufficient to not have gaps, or be more useful than the Smart Edition?

Part of #4869