Skip to content

Current path range should be respected when path to hash and path to KV indices are restored #18571

Closed
@artemananiev

Description

@artemananiev

MerkleDb has a feature to restore path to hash and path to KV indices on load, if the indices are not available on disk. This is done by iterating over all data files, loading all data (hash or KV) records, and updating corresponding index entries.

There are two issues about this feature:

  1. This mode is turned on, when the index is empty after it's loaded from disk:
        final boolean needRestorePathToDiskLocationLeafNodes = pathToDiskLocationLeafNodes.size() == 0;

In addition to this check, first/last leaf paths should be checked, too. If they are -1 (which means the database is empty), there is no need to restore anything.

  1. Data files may be old and may contain entries that are outside of the current path range. It leads to an assertion error in AbstractLongList.putimpl(), which is called from loading callbacks:
            leafRecordLoadedCallback = (dataLocation, leafData) -> {
                final VirtualLeafBytes leafBytes = VirtualLeafBytes.parseFrom(leafData);
                pathToDiskLocationLeafNodes.put(leafBytes.path(), dataLocation);
            };

There is no need to put() if the record path is not within first/last leaf path range.

Metadata

Metadata

Assignees

Type

Projects

Status

✅ Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions