Skip to content

[ENH] Make snapshot be recursive. #4085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 10, 2025
Merged

Conversation

rescrv
Copy link
Contributor

@rescrv rescrv commented Mar 25, 2025

Before this PR, the status quo was to stop at a depth of 2. This allows
for a small-ish log. With this change, the snapshots can be of
arbitrary depth, although in practice they will be log(n)/log(1846)
levels deep.

The motivation for this patch is the realization that 1846**2 gives just under 4 days of logging before the manifest begins to grow again.

Copy link

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

@rescrv rescrv requested review from HammadB and sanketkedia March 25, 2025 22:42
@rescrv rescrv force-pushed the rescrv/wal3-get-to-compact branch 2 times, most recently from 09cf40d to f9750a4 Compare March 31, 2025 16:04
@rescrv rescrv force-pushed the rescrv/wal3-recursive-snapshot branch from 24e75c9 to fae1096 Compare March 31, 2025 18:04
Copy link
Collaborator

@HammadB HammadB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you discover something empirically mandating the need for recursion? I remember the choice to limit the depth being quite deliberate

@rescrv rescrv force-pushed the rescrv/wal3-get-to-compact branch 5 times, most recently from 2b692be to 64470ef Compare April 3, 2025 23:19
Base automatically changed from rescrv/wal3-get-to-compact to main April 4, 2025 03:28
Before this PR, the status quo was to stop at a depth of 2.  This allows
for a small-ish log.  With this change, the snapshots can be of
arbitrary depth, although in practice they will be log(n)/log(1846)
levels deep.
@rescrv rescrv force-pushed the rescrv/wal3-recursive-snapshot branch from fae1096 to 0033895 Compare April 4, 2025 12:31
@rescrv rescrv merged commit 2a24b23 into main Apr 10, 2025
83 checks passed
@rescrv rescrv deleted the rescrv/wal3-recursive-snapshot branch April 10, 2025 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants