Models for the Google Summer of Code 2023 prerequisite task

[adrianboguszewski]

The models listed below are for the GSoC 2023 prerequisite task only.

We provide several potential candidates. Please select only one which hasn't already been selected (look at the checkboxes and comments below). When you decide, assign a model to you by adding a comment with the model name. Then we will tick it to mark reserved.

If you struggle, you can reassign yourself to another non-taken model. However, we can do it only once.

When you create a PR, please follow the self-checklist below:

• each function is described by docstrings and type hints
• notebook contains explicit descriptions and explanatory diagrams
• the notebook doesn't use any data (image, video, etc.) that is not CC4.0 licensed
• there is a README.md file in consistent style (look at other notebooks)
• the notebook is added to the main README
• there are no grammar, punctuation or typo issues (use any free tool for that e.g. Grammarly)
• there are no committed files besides notebook and readme (please use images or videos from data dir)
• your PR doesn't change any other notebooks
• all CI checks passed

Object detection:

• Yolov6 - https://github.com/meituan/YOLOv6 (@ahmd-nish)
• DAMO YOLO - https://github.com/tinyvision/DAMO-YOLO (@Muskan33)
• YoloX - https://github.com/Megvii-BaseDetection/YOLOX (@sawradip)
• RTMDet - https://github.com/open-mmlab/mmyolo/tree/main/configs/rtmdet (@AnuragMaiti)
• EfficientDet - https://github.com/google/automl/tree/master/efficientdet (@ashish-2005)
• CenterNet - https://github.com/xingyizhou/CenterNet/ (@rajuptvs)
• SSD MobileNet V2 http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md (@AlexFierro9)
• FasterRCNN Inception ResNet v2 http://download.tensorflow.org/models/object_detection/tf2/20200711/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8.tar.gz https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md (@Paulooh007)
• YOLOS https://huggingface.co/hustvl/yolos-tiny (@SandeepaDevin)
• DETR https://huggingface.co/facebook/detr-resnet-50 (@Tatwansh)
• YoloR https://github.com/WongKinYiu/yolor (@18yz153)
• YoloF https://github.com/megvii-model/YOLOF (@thegeek13242)
• NanoDet https://github.com/RangiLyu/nanodet (@sahilpmehra)
• UltraFace https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB (@JacketChenlll)
• YoloV7 Face https://github.com/derronqi/yolov7-face (@lucifertrj)
• yolov5-blazeface https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/205-vision-background-removal (@AnuragTimilsina)
• RetinaFace https://github.com/biubug6/Pytorch_Retinaface (@VaillaRohit)

Rotated object detection:

• Rotated FCOS - https://github.com/open-mmlab/mmrotate/blob/main/configs/rotated_fcos/README.md
• ReDet - https://github.com/open-mmlab/mmrotate/blob/main/configs/redet/README.md (@nischay7)
• Roi_trans - https://github.com/open-mmlab/mmrotate/blob/main/configs/roi_trans/README.md

Semantic Segmentation:

• SegFormer - https://huggingface.co/nvidia/segformer-b0-finetuned-ade-512-512 (@Kasliwal17)
• ClipSeg - https://huggingface.co/CIDAS/clipseg-rd64-refined (@RishithaR-388)
• SETR - https://github.com/fudan-zvg/SETR (@AniketARS)
• BeIT - https://huggingface.co/microsoft/beit-base-finetuned-ade-640-640 (@hadyy17)
• Segmenter - https://github.com/open-mmlab/mmsegmentation/tree/master/configs/segmenter (@blaz-r)
• DeepLab V3 - https://github.com/tensorflow/models/tree/master/research/deeplab (@chaitravi-ce)
• FaceParsing |& MakeUp - https://github.com/zllrunning/face-parsing.PyTorch https://github.com/zllrunning/face-makeup.PyTorch (@Lj1ang)
• ESPNet https://github.com/sacmehta/ESPNet (@Nouran-Muhammad)
• YoloP https://github.com/hustvl/YOLOP

Instance Segmentation:

• YOLACT https://github.com/dbolya/yolact.git (@Abdullah-Elkasaby)
• Mask RCNN Inception ResNet V2 http://download.tensorflow.org/models/object_detection/tf2/20200711/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.tar.gz https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md (@mr-rajashekhar)

Action/Gesture recognition:

• TSM - https://github.com/mit-han-lab/temporal-shift-module (@ntombi)
• Timesformer - https://huggingface.co/facebook/timesformer-base-finetuned-k400 (@BrennoMello)
• SlowFast - https://github.com/open-mmlab/mmaction2/blob/master/configs/recognition/slowfast/README.md (@rajatkrishna)
• YOWOv2 - https://github.com/yjh0410/YOWOv2 (@Matrixmang0)
• movinet - https://github.com/tensorflow/models/tree/master/official/projects/movinet (@sharvesh642)
• xclip https://huggingface.co/microsoft/xclip-base-patch32

Background matting:

• ModNet - https://github.com/ZHKKKe/MODNet
• Robust Video Background matting - https://github.com/PeterL1n/RobustVideoMatting (@wulongjian)
• MGMatting https://github.com/yucornetto/MGMatting
• PortraitNet https://github.com/dong-x16/PortraitNet

Old Photos Restoration/Image colorization/Image denoising/super resolution:

• Bringing Old Photos Back to Life - https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life (@Om-Doiphode )
• DeOldify - https://github.com/jantic/DeOldify (@Dhruvanshu-Joshi)
• Coltran - https://github.com/google-research/google-research/tree/master/coltran (@weronikazak)
• Colorizer https://github.com/richzhang/colorization (@pyther-hub)
• SwinIR - https://github.com/JingyunLiang/SwinIR (@Z-Fran)
• style-swapping https://github.com/irasin/Pytorch_Style_Swap (@m-gopichand)
• Real-ESRGAN (for real images) - https://github.com/xinntao/Real-ESRGAN/blob/master/docs/model_zoo.md (@aadhamm)
• Real-ESRGAN (for animation video) - https://github.com/xinntao/Real-ESRGAN/blob/master/docs/model_zoo.md
• RCAN https://github.com/yulunzhang/RCAN
• Super-SlowMo https://github.com/rmalav15/Super-SloMo
• Photo2Cartoon https://github.com/minivision-ai/photo2cartoon (@sususama)

Depth estimation:

• lite-mono https://github.com/noahzn/lite-monoc
• MiDaS 3.1 https://github.com/isl-org/MiDaS (@nsk126)
• Vi-Depth https://github.com/isl-org/VI-Depth (@pronoym99)

Text classification:

• Roberta - https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment @ABHIJATSARARI)
• XLM-Roberta - https://huggingface.co/papluca/xlm-roberta-base-language-detection (@hazrulakmal)
• DepRoBerta - https://huggingface.co/rafalposwiata/deproberta-large-depression (@SpyzzVVarun)
• CodeBerta - https://huggingface.co/huggingface/CodeBERTa-language-id (@zilto)
• Albert V2 - https://huggingface.co/textattack/albert-base-v2-MRPC (@dwipddalal)
• DistilRoberta - https://huggingface.co/j-hartmann/emotion-english-distilroberta-base (@MR-ENVYR)
• FinBERT https://huggingface.co/yiyanghkust/finbert-tone (@shrey-2803)
• Deberta https://huggingface.co/microsoft/deberta-base-mnli (@mhy-666)

Token classification:

• Part of speech tagging - https://huggingface.co/flair/pos-english (@harish2773)
• Punctuation restoring (bert-restore-punctuation) - https://huggingface.co/felflare/bert-restore-punctuation (@seanjyu)
• Punctuation restoring (punctuate-all) - https://huggingface.co/kredor/punctuate-all (@theNobody-12)
• Typo detection - https://huggingface.co/m3hrdadfi/typo-detector-distilbert-en (@Ravindu987)
• Named entity recognition - https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english (@Aditya-vardhan13)

Text generation:

• BioGPT https://huggingface.co/microsoft/biogpt (@sidyakinian)
• gpt-neo https://huggingface.co/EleutherAI/gpt-neo-125M (@Warlord-K)
• OPT https://huggingface.co/facebook/opt-350m (@zhumakhan)

Text Summarization

• DistilBART https://huggingface.co/sshleifer/distilbart-cnn-12-6 (@samycolen)

Question Answering

• MiniLM https://huggingface.co/deepset/minilm-uncased-squad2 (@Akshit17)
• ELECTRA https://huggingface.co/deepset/electra-base-squad2 (@sanjayk0508)
• DistilBert https://huggingface.co/distilbert-base-cased-distilled-squad (@fajemila)

Sound classification:

• speech emotions recognition (wav2vec) https://huggingface.co/harshit345/xlsr-wav2vec-speech-emotion-recognition (@paxF3E)
• Hubert key words spotting https://huggingface.co/superb/hubert-base-superb-ks (@100-87)

[sawradip]

Hello, firstly I want to thank you for the awesome list. I had a question:

Is there any existing instance/example of using open-mmlab models, or such config based model libraries(Ex: detectron2), with Openvino?

[sawradip]

I would like to assign YoloX to myself.

[adrianboguszewski]

@sawradip, assigned to you. We don't have such examples in this repository and I'm not aware of any existing examples of open-mmlab or detectron2 with OpenVINO.

[eaidova]

@sawradip please check https://github.com/open-mmlab/mmdeploy this library support export mmlab pretrained models to OpenVINO format

[sawradip]

Thank you for assigning me that task. I will submit a PR on that, in a few days.

And I am interested on the open-mm projects is, such a good number of models, are out of openvino accessibility. Though It is not GSOC requirement, neither on planned projects, I was thinking, If I could contribute some tutorial, on integrating open-mm or similar config based models.

[akashAD98]

@eaidova should we need to implement any of the repo in openvino ir format? this is only for GSoC people or anyone from open source community can contribute

[eaidova]

@akashAD98, this activity is only for GSoC participants for now (we use this models list for candidates selection), but we are open for community contributions too :)

If you have any ideas or great models for openvino notebooks, feel free to contribute :)

[Kasliwal17]

Hi, I would like to assign SegFormer to myself.

[ahmd-nish]

Hi, i would like to assign YOLOv6 to myslef and contribute. Can you please help me with that ?