free operation queue semaphore in process snapshot chunk and when hitting error in handleSnapshot that prevents us from calling process snapshot chunk #1420
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ting error in handleSnapshot that prevents us from calling process snapshot chunk
…bytes if checksum isn't disabled
diamondhands0
approved these changes
Oct 24, 2024
| @@ -1255,6 +1255,8 @@ func (bc *Blockchain) LatestLocator(tip *BlockNode) []*BlockHash { | |||
| } | |||
|
|
|||
| func (bc *Blockchain) HeaderLocatorWithNodeHash(blockHash *BlockHash) ([]*BlockHash, error) { | |||
| bc.ChainLock.RLock() | |||
There was a problem hiding this comment.
Just give me a comment on why we need the locks here.
lib/server.go
Outdated
| @@ -1658,6 +1687,9 @@ func (srv *Server) _handleSnapshot(pp *Peer, msg *MsgDeSoSnapshotData) { | |||
| "attempt to HyperSync from the beginning. Local db checksum %v; peer's snapshot checksum %v", | |||
| checksumBytes, srv.HyperSyncProgress.SnapshotMetadata.CurrentEpochChecksumBytes))) | |||
| if srv.forceChecksum { | |||
| // Free up a slot in the operationQueueSemaphore, since we hit an error and it won't be freed | |||
| // by processing the snapshot chunk. | |||
| srv.snapshot.FreeOperationQueueSemaphore() | |||
There was a problem hiding this comment.
See my fixme above. Seems like a bug to call it here?
lib/snapshot.go
Outdated
| } | ||
|
|
||
| mainDbMutex.Lock() | ||
| // TODO: I think we don't need to hold the chain lock. We can have multithreaded writes. |
There was a problem hiding this comment.
I believe this is correct. Key-value pairs in snapshot chunks should never overlap, and we never fetch the same snapshot more than once so I think this is OK. The lock was probably put here to be extra cautious.
lib/server.go
Outdated
| @@ -1441,13 +1441,16 @@ func (srv *Server) _handleSnapshot(pp *Peer, msg *MsgDeSoSnapshotData) { | |||
| msg.SnapshotChunk[0].Key, msg.SnapshotChunk[len(msg.SnapshotChunk)-1].Key, len(msg.SnapshotChunk), | |||
| msg.SnapshotMetadata, msg.SnapshotChunk[0].IsEmpty(), pp))) | |||
| // Free up a slot in the operationQueueSemaphore, now that a chunk has been processed. | |||
| srv.snapshot.FreeOperationQueueSemaphore() | |||
| // srv.snapshot.FreeOperationQueueSemaphore() | |||
There was a problem hiding this comment.
I left some comments in this file as FIXMEs because I couldn't comment on the lines that were unchanged in the PR. See my edited PR here:
There was a problem hiding this comment.
I also tried a weird alternative with defer that may be less error-prone. Check it out up to you if you want to change it.
…1423) * make GetUtxoViewAndUtxoOpsAtBlockHash public (#1422) * review * remove fixmes and add comments --------- Co-authored-by: Lazy Nina <[email protected]> Co-authored-by: Lazy Nina <[email protected]>
lazynina
deleted the
ln/free-operation-queue-semaphore-in-process-chunk
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.