Friday, April 21, 2017

DL 101

1. A friendly intro to neural nets by Karpathy (Link)

2. Get started with DL coding with bare essentials, w/o worrying about AWS setup or theory (Link)

3. Now do worry about AWS setup (video, scripts)

4. Intro to image classification  - Nearest Neighbour classifier, Linear classifier

5. Get running keras image classification script (dogs vs cats competition) under AWS setup (PDLC_Lec_1, PDLC_Lec_1_notebook)

6. More on dogs_vs_cats project and a coding intro to neural nets and gradient descent (PDLC_Lec_2, PDLC_Lec_2_notebook)

7. A concept-heavy deep dive into loss functions and gradient descent (Link)

8. A concept-heavy intro to Backprop and Activation functions (Link)

9. A concept-heavy intro to Neural network architecture (Link)

10. A coding-heavy intro to 'Convolutional' Neural network architecture (PDLC_Lec_3, PDLC_Lec_3_notebook)

11. A concept-heavy intro to 'Convolutional' neural network architecture: Link1, Link2, Tranfer Learning

12. Practical advice on data-preprocessing, Regularization, Loss function intuition, batch normalization (Link)

13. Optimization revisited, Statefarm competition, semi-supervised learning, collaborative filtering PDLC_Lec_4, PDLC_Lec_4_notebook

14. Practical advice on parameter tuning, learning rates, model ensembling. (Link)

15. Code-heavy guide to batch normalization, intro to NLP, intro to RNN (PDLC_Lec_5, PDLC_Lec_5_notebook)

16. Implement your own neural network from scratch (Link)

17. RNN continued, Theano (PDLC_Lec_6, PDLC_Lec_6_notebook)

18. Concept-heavy RNN by Karpathy (Link)

19. Coding-heavy CNN 2.0, Advanced RNN with GRU and LSTM (mainly GRU), Fisheries competition (PDLC_Lec_7, PDLC_Lec_7_notebook)


Path I intend to follow, before setting out to do anything: 1, 2, 3, 4, ... 18, 19

Path I actually follow (will keep editing this as I go along):

1, first half of 2, 4

After DL101:

DL102: Part 2 of the FastAI course
DL103: Generative Adversarial networks (Link)
DL104: + Karpathy post on RL + OpenAI Gym