Remove an unwanted potential host sync in operators taking pinned inputs #5822
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…uts. Signed-off-by: Michał Zientkiewicz <[email protected]>
mzient
changed the title
|
CI MESSAGE: [24083082]: BUILD STARTED |
jantonguirao
approved these changes
Feb 17, 2025
|
CI MESSAGE: [24083083]: BUILD STARTED |
JanuszL
approved these changes
Feb 17, 2025
|
CI MESSAGE: [24083083]: BUILD FAILED |
|
CI MESSAGE: [24083082]: BUILD FAILED |
|
CI MESSAGE: [24083083]: BUILD PASSED |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Category:
Bug fix (non-breaking change which fixes an issue)
Description:
The change #5820 fixed a race condition, but introduced a possibility of host synchronization with the (just scheduled) operator. This would cause unwanted bubbles in GPU activity.
Currently we don't know whether a pinned input is consumed in host order or device order. For that reason, we need to wait for the operator's completion event in the input's associated deletion order.
This PR works around that problem by synchronizing the host with the input stream to enable host-side access, but assigning the workspace output stream to the input, so we don't wait on host for the operator to complete its GPU work.
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Tests:
test_pipeline.test_gpu2cpu2mixedChecklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A