Best practices to change downstream task?

[rsousa95]

Hi all,

I found the timm library and it has a lot of models. However, most of them are ready for classification tasks. Based on this guide, I understand it's possible to use models as feature extractors for other tasks.

However, my challenge is how to integrate these feature extractors into object detection pipelines. Specifically, while timm provides a lot of backbones, there doesn't seem to exists a dedicated library or module for detection heads that can easily connect with these backbones. I've looked at other frameworks like MMDet, Detectron2 or Ultralytics, but they seem to come with their own set of models and configurations.

How could I do this? Something like trying to attach the later components of FasterRCNN and getting them to match?

So, can anyone recommend approaches for integrating timm feature extractors into object detection models? Is there a library or resource that offers modular detection heads compatible with timm backbones that I may have missed? Can someone share an example of integrated timm with other tasks beside classification?

Thank you!

Just for extra context: what I'm actually trying to do is some sort of self-supervised learning. I'd like to train a backbone model on a pretext classification task. And then use that trained model and adapt it for another task. I'm mostly interested in object detection but segmentation would also be an option. So I guess I need the backbone to be compatible with every task.

[rwightman]

The efficientdet lib I created some time ago was meant to serve as an example, it's not actively added to, but it still works and I fix issues as they come up: https://github.com/rwightman/efficientdet-pytorch

Segmentation models is actively updated and uses timm models as encoders https://github.com/qubvel-org/segmentation_models.pytorch

I'm sure others can be found, I ran into one recently but need to refresh memory

[rwightman]

Oh yeah, and mmdetection has a timm example:

• https://github.com/open-mmlab/mmdetection/tree/cfd5d3a985b0249de009b67d04f37263e11cdf3d/configs/timm_example