Assignments From Google's Deep Learning Class

A quick note for anyone interested in deep learning - I've been working through Google's deep learning class on Udacity and posted (most of) the assignments on my IPython Github repo. There are links in the README to rendered versions of the notebooks. I said I posted "most of" the assignments because I haven't finished the last one yet, and it may be a while before I circle back to it. It turns out that implementing recurrent nets and sequence-to-sequence models from scratch using low level primitives, with no prior experience or theoretical foundation, is pretty fucking hard =) I fully intend to work on it, I just need to spend a lot more time studying and playing around with the code.

I'm planning to do a future blog post where I'll go more in depth on this class, and deep learning in general, but for now I'll just offer a few initial impressions of the class in case anyone else is planning to give it a try.

The video content, while informative, is very light on substance. After doing a number of MOOCs on Coursera, I was a bit disappointed by the lack of depth in Udacity's course format. I get that a lot of the Coursera content is developed by universities and adapted from real semester-long courses, and by contrast, many Udacity courses (such as Google's course) are developed by companies with a somewhat limited investment in the curriculum. But there's a pretty stark difference between the depth of a class like Andrew Ng's machine learning class and this one.

As a direct consequence of this, don't expect to learn everything you need to complete the assignments from watching the videos. You really need to be willing to go out and read research papers on relevant topics to meaningfully progress on the labs. If that's not your thing then you can just watch the videos and skip the assignments but all of the "meat" of the class is in the notebook code posted with the labs. You're expected to be reasonably comfortable building fairly complex computation graphs in Tensorflow (or at least comfortable enough to learn as you go). The course itself doesn't teach you any of this so you'll have to seek that knowledge out yourself.

If you can get past those hurdles, there's quite a lot of value in the code that Google provides with the labs. I found it to be fairly complex and hard to follow at times, but if you're willing to invest the time there's a lot to learn. It's amazing how much effort can go into just pre-processing, formatting, and batching data to get it ready for input to a neural net, and there are some great of tips and tricks to learn from the code. Likewise, the example code in Tensorflow is very helpful and sets a good foundation to build on with the labs.

Overall I enjoyed the class but wouldn't recommend it as a first step into deep learning. It seems much better suited for individuals with prior conceptual knowledge of the subjects that are looking to develop hands-on experience. If you're totally unfamilar with deep learning and just looking to get started, I'd recommend some of the free online books linked in Awesome Deep Learning as a starting point.