Support for CLIP

[Sidd1609]

Hello, wanted to say that the repo is really well put and maintained! really appreciate everyone's effort.

I am currently trying to visualize concept activations on the image features using model = "ViT-L/14" for CLIP.
I figured the target_layers but I am getting the issue where I need to also pass the text_features as:

ine 1501, in _call_impl
return forward_call(*args, **kwargs)
TypeError: forward() missing 1 required positional argument: 'text'

is there anyway of passing arguments to the forward call for the loaded model?

Thanks
Regards
-SriSiddarthC

[jacobgil]

Hi @Sidd1609 ,

You can create a wrapper around the model that will call it with the desired signature.

[FUIGUIMURONG]

Hello, wanted to say that the repo is really well put and maintained! really appreciate everyone's effort.

I am currently trying to visualize concept activations on the image features using model = "ViT-L/14" for CLIP. I figured the target_layers but I am getting the issue where I need to also pass the text_features as:

ine 1501, in _call_impl return forward_call(*args, **kwargs) TypeError: forward() missing 1 required positional argument: 'text'

is there anyway of passing arguments to the forward call for the loaded model?

Thanks Regards -SriSiddarthC

I also want to visualize the clip visual feature, do you slove it ?

[Sidd1609]

No not yet

…

On Mon, Nov 27, 2023 at 11:13 AM FUIGUIMURONG ***@***.***> wrote: Hello, wanted to say that the repo is really well put and maintained! really appreciate everyone's effort. I am currently trying to visualize concept activations on the image features using model = "ViT-L/14" for CLIP. I figured the target_layers but I am getting the issue where I need to also pass the text_features as: ine 1501, in _call_impl return forward_call(*args, **kwargs) TypeError: forward() missing 1 required positional argument: 'text' is there anyway of passing arguments to the forward call for the loaded model? Thanks Regards -SriSiddarthC I also want to visualize the clip visual feature, do you slove it ? — Reply to this email directly, view it on GitHub <#460 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALTC7VZSWRNMUF32LLWH6HDYGQR7JAVCNFSM6AAAAAA5E6TJROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRXGE2TQMJQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Danny-1-8]

Hello, wanted to say that the repo is really well put and maintained! really appreciate everyone's effort.

I am currently trying to visualize concept activations on the image features using model = "ViT-L/14" for CLIP. I figured the target_layers but I am getting the issue where I need to also pass the text_features as:

ine 1501, in _call_impl return forward_call(*args, **kwargs) TypeError: forward() missing 1 required positional argument: 'text'

is there anyway of passing arguments to the forward call for the loaded model?

Thanks Regards -SriSiddarthC

Hello friend,do you think the "cam" is compatible with the visual encoder of CLIP?

[Aashish75]

@Sidd1609 @Danny-1-8 @FUIGUIMURONG were any of you able to get past the "TypeError: forward() missing 1 required positional argument: 'text'" and visualise the activations on the image?

[jzousz]

I also want to visualize the clip visual feature.

[jackyjinjing]

I tried it and it worked successfully，I'll try to add an example of a clip.

[snow-like-kk]

I tried it and it worked successfully，I'll try to add an example of a clip.

Hi，I'm new in this field, can I ask u about the effect of function "reshape_transform"? I try to modify the code for annother vlm model, but there are some questions with the function. I guess the error is relevant to the model size.