reconcile: support "merge" in objects

[hyperknot]

A common pattern I believe is to reconcile partial state updates in stores.

For example, part of a store is synced to a backend, so after a push-to-backend operation, it returns an up-to-date data for part of the store values. Other values, like UI related ones are not included in this data.

Passing this raw data into setStore works, but triggers unnecessary updates. It triggers every single key's update, which is an array or object.

Here is my 1st idea to tackle this: running reconcile in a for loop on each 1st level key:

for (const key of Object.keys(data)) {
  this.setState(key as any, reconcile(data[key as keyof typeof data]))
}

This works, and when put in a batch() should be performant. Quite a lot of code, so asks for a wrapper function, but should be good.

My 2nd idea is this:

this.setState(reconcile({ ...this.state, ...data }))

This also works. If I understand correctly, it's a tiny bit slower, but the slowness happens in the pure JS code, not in the actual UI updates, so it should still be fast.

When I looked at the source code for reconcile and found that it supports merge, but only for arrays.

I'd like to recommend to add support for merge in objects. To make sure that this does not modify the array behaviours in sub-keys, it should be called something different, "partial" probably.

I believe it'd be as simple as turning off the delete loop in the first level of applyState.

Please tell me if I'm not correct in this. Isn't this a common pattern when using stores?

How do you normally solve this?

[hyperknot]

I did some performance testing on my realistic workload.

spread - 0.275ms

      this.setState(reconcile({ ...this.state, ...data }))

for loop no batch - 0.205ms

        for (const key of Object.keys(data)) {
          this.setState(key as any, reconcile(data[key as keyof typeof data]))
        }

for loop with batch - 0.225ms

      batch(() => {
        for (const key of Object.keys(data)) {
          this.setState(key as any, reconcile(data[key as keyof typeof data]))
        }
      })

I think the difference is so extremely small, that I'll stick with the spread as it has the cleanest syntax.
It's interesting that batch is more, but I only run 20 times and it can easily be just a measurement error.

---

I also did a run in a 10k for loop, and got 370 ms for spread and 240 ms for batch and no-batch. I mean that's for a 10k loop!

Basically this is so extremely fast for a single call, that there is no point of optimising for performance.

[ryansolid]

I agree this is sort of loose territory. We have some big change to store reconcile in 2.0. Mainly there is no merge option. Right now we merge things sometimes and others not and it isn't really consistent. My current prototype treats everything as merge unless it finds a key mismatch and then it swaps. And we provide the ability to provide a function based key so different "models" in the store can have different keys if necessary.

The delete loop is necessary for clearing stores... it's actually the only way to reset them. That being said there are more internal changes in reconcile coming in this as well. It's unlikely we change anything in 1.0. As you said you can reconcile deeper along the path or clone if you want shallowly which is inexpensive.

[hyperknot]

I'm happy to wait for 2.0 to have some improvements. The other common issue is Date() objects.

Right now, setStore + reconcile handles every common types (both basic and wrappable ones), except Date() objects.

There is no way to set a Date() object without triggering an update.

So, for both of these cases, I ended up making this utility function for my personal use. It does partial reconcile and proper Date comparison, using getTime().

export function setStateSmart(store: any, data: Record<string, any>): void {
  batch(() => {
    for (const [key, newValue] of Object.entries(data)) {
      // smart Date() comparison
      if (newValue instanceof Date) {
        const oldValue = store.state[key]
        if (oldValue instanceof Date && oldValue.getTime() === newValue.getTime()) continue
        store.setState(key, newValue)
        continue
      }

      // partial reconcile for the rest
      store.setState(key, reconcile(newValue))
    }
  })
}

[ryansolid]

Yeah for non-primitive values it is going to look at the reference. I'm not sure if there is much anything to do for that. We've been looking at wrapping anything you stick in the store by default.. but that isn't going to help here I think because again new instance of date.
I can't think of a really great way to do custom diff mid reconcile. I guess it could be like the key function. It's just we only compare keys and then we merge.. so there is no merge of custom objects like this only replace which would trigger even if the key were the same.
Yeah I don't really see a straightforward solution for this. You'd almost want some like well structured store solution. Sort of like MobX state tree at that point which is I think way beyond the base primitive capability.

[hyperknot]

What about adding a special case for Date()? With Date() you'd cover 99% of the most common types.

Or, adding a type -> eqFn map like {Date: 'getTime') or similar. Then it could be extended for any kind of custom type.

For the partial part, I think reconcile/setState needs to make a difference between 1st level and 2+ levels. Right now it's a beautiful solution, but it doesn't handle the most basic use case very well, that is to differentiate between the store object and a subkey.

I mean for a subkey, you definitely want deletion, but for the full store object, it's very common to do partial updates. I recommend some kind of differentiation for store vs. subkey in the 2.0 version.