There are a lot of great resources on the web that contain implementations of deep learning architectures, but they can be a little hard to find. This post aims to highlight and categorize some of the best resources that I have found. Most repositories have either Tensorflow or PyTorch implementations, so this post is divided by frameworks.
Table of contents
- Framework-agnostic Resources
- Framework-specific Resources
Papers with Code
Although most places have code for a specific framework, one of the best places to find code has implementations for every framework. It’s called Papers with Code. This is absolutely the best way to find implementations in places that you wouldn’t normally look. For example, if you go to the page for RetinaNet, it shows the excellent fizyr implementation of RetinaNet, which I wouldn’t have found otherwise. It ranks all the implementations by GitHub stars, so it’s easy to find some good ones. As well as PyTorch and Tensorflow, it also often has implementations for MXNet.
There is also a Model Zoo website that contains implementations of every type from with every framework. There’s a lot of great variety here.
The first place to go to find Tensorflow implementations is probably the Tensorflow models repo. There’s tons of hidden stuff inside this repo. For starters, I would direct people to the
research folder. There are a lot of implementations there, and, within that, there’s another set of models in the slim directory. Also inside the
research directory is a section on keras models that has implementations of many well-known models, such as inception, mobilenet, and resnet.
If you’re particularly interested in object detection, tensorpack is a great place to look. If you don’t know where to start, take a look at the
examples folder. Also, take a look at the TensorFlow Object Detection API.
PyTorch doesn’t have as many pre-trained models as Tensorflow but there’s still a lot there. If you’re interested in computer vision Torchvision Models is a good place to start.
The PyTorch Image Models repo has lots of different models in it, with more added all the time. Very up-to-date. You can also install this with
pip install timm.
The best place to look for PyTorch object detection models is certainly Facebook’s Detectron2 repository. There’s a lot in here but if you’re looking for pre-trained networks to download for use or transfer learning, definitely check out the Model Zoo. Another great repo for object detection in PyTorch is mmDetection.
Finding code for FastAI is tricky at the moment because 1. the user base is much, much smaller than either Tensorflow or PyTorch (although PyTorch code is compatible, it’s just not smooth) and 2. the code is still in a state of rapid change. Most code from a few years ago will be very difficult to run on today’s FastAI codebase (and fastai2 is coming along soon). However, if you are watching the FastAI videos, you can follow along with the notebooks used in the latest course. The notebooks have changed so you may have to tweak things but it’s a good place to get started.
I have found that object detection resources are the most lacking, so I’ve tried to focus on those. Most of the code is in notebooks due to the teaching style of the course. The most recent official object detection notebook I’ve found is from version 3 of the course. It does basic object detection and has a video lecture associated with it.
There’s also a notebook on multiclass object detection, although it’s even older and I don’t believe is included in the courses anymore. Both of these notebooks use the PASCAL VOC 2007 dataset. There’s also a video to go along with it.
Outside of that, some other notebooks in the repository are worth looking at. This notebook is more recent than the others, but it’s not officially part of the course (as far as I can tell). And although FastAI2 isn’t ready yet, there are a lot of good notebooks in the repo.
If you want to find the latest information on object detection notebooks in FastAI, this thread is a good place to look.
There are also student notebooks that are worth looking at. This student implemented RetinaNet using FastAI. The code is out of date with the most recent version of FastAI but it’s still worth looking at. There’s an implementation of SSD here, although I haven’t had a chance to go through it yet. This repo used FastAI for Whale Identification. These notebooks have instance segmentation, which I haven’t been able to find much of.
I would like to create my own implementations of some of these object detection algorithms using FastAI and update this post with them, but I’m not sure when I’ll get to that.