You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When evaluating log probabilities for the DOMIAS metric, gradient computation is not disabled, causing massive overhead when the test set is large.
During training, log probabilities are computed in batches. However, during inference this is not the case, causing massive overhead since gradients are still being computed.
How to Reproduce
Go to 'synthcity/metrics/eval_privacy.py', line 594. Here we see that log probabilities are computed for the entire test set at once, without disabling gradients. When the test set is made to be reasonably large, this (unnecessarily) takes up a large amount of memory.
Expected Behavior
Disable gradient computation during inference when computing log probabilities for DOMIAS.
The text was updated successfully, but these errors were encountered:
Description
When evaluating log probabilities for the DOMIAS metric, gradient computation is not disabled, causing massive overhead when the test set is large.
During training, log probabilities are computed in batches. However, during inference this is not the case, causing massive overhead since gradients are still being computed.
How to Reproduce
Expected Behavior
Disable gradient computation during inference when computing log probabilities for DOMIAS.
The text was updated successfully, but these errors were encountered: