[Bug] TypeError: __init__() got an unexpected keyword argument 'features_extractor_class'

[watermeloncq]

Hello araffin, I follow your instruction to costomizing policy network for TD3 algorithm #425 , after I defined the TD3 model and run it, errors happens.
(By the way, I am not a girl, but a man)

The code of defining TD3 model

model = TD3("CustomTD3Policy", 
            env, 
            features_extractor_class=CustomCNN,
            features_extractor_kwargs=dict(features_dim=3072), 
            verbose=1)

🐛 Bug

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-40f9b6acf8d0> in <module>
----> 1 model = TD3("CustomTD3Policy", 
      2             env,
      3             features_extractor_class=CustomCNN,
      4             features_extractor_kwargs=dict(features_dim=3072),
      5             verbose=1)

TypeError: __init__() got an unexpected keyword argument 'features_extractor_class'

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[Miffyli]

You need to pass the feature extractor in policy kwargs. See docs.

[watermeloncq]

You need to pass the feature extractor in policy kwargs. See docs.

ok, I follow your guide, but another error happens.

Here is my code:

policy_kwargs = dict(
    features_extractor_class=CustomCNN,
    features_extractor_kwargs=dict(features_dim=3072),
    action_noise=action_noise
)
model = TD3("CustomTD3Policy", env, policy_kwargs=policy_kwargs, verbose=1)

After I run my code, the another error occurs:

<ipython-input-16-5baf8dded4fd> in __init__(self, *args, **kwargs)
     81 class CustomTD3Policy(TD3Policy):
     82     def __init__(self, *args, **kwargs):
---> 83         super(CustomTD3Policy, self).__init__(*args, **kwargs)
     84 
     85     def make_actor(self, features_extractor: Optional[BaseFeaturesExtractor] = None) -> CustomActor:

TypeError: __init__() got an unexpected keyword argument 'action_noise'

I am almost exhousted, could I upload my code to you for debugging?

[Miffyli]

There is no parameter action_noise in TD3 policy. I am sorry to say but we do not have time to offer tech support on individual basis. These issues are for reporting bugs and proposing enhancements.

[watermeloncq]

There is no parameter action_noise in TD3 policy.

But I see the user guide of TD3 have the action_noise parameter

[araffin]

But I see the user guide of TD3 have the action_noise parameter

Please read the documentation carefully, action_noise is a parameter to the TD3 class, the same goes for policy_kwargs, features_extractor_class is a parameter of the policy class, not TD3 constructor, hence it must be passed to the policy using the policy_kwargs dictionary.

[watermeloncq]

Thank you, I am sorry that I have so many question. My major is finance, and I am not a professional programmer, I try to find a partner to help me migrate my paper code to sb3, but no one can help me.
Would you please tell me which forum that professionally deal with the problem of usbing sb3?

[Miffyli]

See the beginning of the question issue template for links to such forums.