Custom Policy Network - Off-Policy Algorithm - Advanced Customization

[Webbah]

📚 Documentation

As stated in the documentation, for using a Custom Policy Network for off-policy algorithms #425 is linked which shows an example of how to do this.
At the end register_policy("CustomTD3Policy", CustomTD3Policy) is used to use the Custom Policy e.g. for TD3/DDPG.
Since it looks like, that in the commit 254bb10 the register_policy-function was deleted/replaced, how can this be handled with the current version of SB3?

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation

[qgallouedec]

Thank you for reporting. We will update the documentation.

Use the following syntax:

from stable_baselines3 import TD3
from stable_baselines3.td3.policies import TD3Policy


class CustomTD3Policy(TD3Policy):
    ...


TD3(CustomTD3Policy, "MountainCarContinuous-v0").learn(5_000)

[Webbah]

Thank you for the quick answer, works for me!
I would close the issue, which e.g. could be linked in the documentation.

[qgallouedec]

Reopening to remember to change the documentation.

[araffin]

Hello,
if you want the same feature as the register policy, the actual syntax is:

from stable_baselines3 import TD3
from stable_baselines3.td3.policies import TD3Policy

TD3.policy_aliases["CustomTD3Policy"] = TD3Policy

TD3("CustomTD3Policy", "MountainCarContinuous-v0").learn(5_000)

I'll update the issue

[Webbah]

Great, thank you!