Introduction to Deep Learning

Winter Semester 2019
Prof. Dr. Sebastian Stober, Jens Johannsmeier & Andreas Krug
Mon 11-13 c.t.; House 29 - Lecture Hall 307

Start: Monday, October 14

“I have worked all my life in Machine Learning, and I’ve never seen one algorithm knock over benchmarks like Deep Learning.”
Andrew Ng (Stanford / Baidu)

Over the last decade, so-called “deep learning” techniques have become very popular in various application domains such as computer vision, automatic speech recognition, natural language processing, and bioinformatics where they produce state-of-the-art results on various challenging tasks. Most recently, deep neural networks even succeeded in mastering the game of go.

A crucial success factor of deep neural networks is their ability to learn hierarchies of increasingly complex features from raw input data. Such representations often outperform traditionally hand-crafted features that require expensive human effort and expertise. But they do not come as a free lunch. Designing and training deep neural networks such that they actually learn meaningful and useful feature representations of data is an art itself. Mastering it requires practice and experience - and this course offers many opportunities to obtain these.

This is an intense course following a “flipped classroom” design that builds on everybody’s active participation. The curriculum roughly follows Part II of the Deep Learning Book but also covers recently published advances in the field. You will be responsible to prepare for each class by reading selected literature or watching online video lectures and talks. Student are further encouraged to self-reflect on their learning progress in a brief weekly blog post. This can, for instance, include open questions, self-discovered answers or pointers to additional helpful material. Most of the in-class time will be used to discuss the open questions about the concepts under study. There will also be increasingly complex programming assignments where you can apply and practice what you just learned and develop new ideas. All implementations will be in Python (version 3) using Tensorflow as base framework within the Google Colaboratory environment.

In your final course project, you will implement a deep learning architecture for a given task yourself. This will be a collaborative team effort with a little competition about which team eventually obtains the best results. For this, you will have access to professional-grade GPU compute servers. Each team will document their progress in blog posts and describe their final solution in a short paper.

As prerequisites for the course, you need to already have basic knowledge on machine learning in general as well as linear algebra and probability theory. Those who would like a refresher prior to taking this class can find an excellent compilation in Part I of the Deep Learning Book. You also have to know already how to program in Python at least a basic level. For a quick introduction, several tutorials are linked from the official Python wiki. And of course you should bring some curiosity to embark on an open-ended and exploratory learning process.

The number of participants for this course is limited to 60. Admission is by application as detailed below.

Administrative Terms & Conditions (a.k.a. the “fine print”)

The following figure gives an overview about the three possible prerequisites and the corresponding application requirements.
Please take a moment to read these information carefully!
If you afterwards have got further questions about the course, please contact Sebastian Stober.

(Link to LSF system, mail to Sebastian Stober)
Your submitted information will be solely used for application process. It will be treated confidentially and deleted after the candidates have been selected.

Application process points 1 and 2 have to be finished by October 7 (23:59). Applications arriving later will only be considered if there is still course capacity.

The aim of this process is to select students that are likely to sucessfully complete the course and not drop out half way through.

Credits

The course belongs the to topic domain “practical computer science” and amounts to 6 credit points or an equivalent of 180 hours. The expected weekly commitment is about 10 hours. Be prepared to spend that much time! If you are into the topic and want to go deeper, rather plan for some extra time - it’s worth it!

Exams

The lecture concludes with a two-hour written exam. Students can decide whether they want the exam to be a written exam or a (un)graded certificate of achievement (Schein). Admission to the exam must be obtained through active participation throughout the course (especially in the exercises, see below).

Students who need an ungraded certificate of achievement must also attend the exam and gain admission through active participation in the course. Upon passing the exam (grade 4.0 and better) the certificate will be issued.

Exam Admission Criteria

In order to be admitted to the exam, all of the following criteria have to be met:

At least 80% of the weekly assignments completed in time (including being able to present the solution on request during the respective exercise session)
Significant contribution to the course project (at least one blog post and progress report) and the resulting group paper

Exercises

To participate in the exercise, you must register for one of the two exercise groups. Registration is possible October 7–11 (23:59), 2019. To register for an exercise group, please use the respective function in the LSF system. When you apply for the exercise groups, you can specify priorities for the different dates. Please specify several dates with different priorities as we cannot guarantee that everybody will be assigned to their first choice!

Please note: The final grouping of students into exercise groups will take place after the LSF registration deadline has passed. The allocation is based on the priorities that you indicated when you signed up.

The change or visit of another group should remain the exception and must be communicated by email before the first of the two exercises.

Programming Assignments & Course Project

For roughly the first two thirds of the lecture, there will be weekly programming assignments. Solutions can be handed in until a specified deadline and will be discussed in the subsequent exercise session. They do not have to be completely correct, but it must be clearly visible that a substantial effort has been made to solve the task. Assignments can be done in groups of up to three students, but each student has to be able to present and explain their solution by themselves if asked to. If a solution contains copied / adapted code, the respective sources need to be cited in the code comments. If you fail to explain your solution (which should not happen if it is your solution), the assignment will be considered as failed. Two fails will result in exclusion from the exam.

At least 80% of the assignments need to be handed in. Please plan accordingly and work in advance if you know you will be absent during the semester! Exception can only be made in case of illness (which requires a medical certificate from a doctor).

For the remaining third of the course, practical work will focus on the course project. The exercise groups will compete for the best solution and split up into smaller team to investigate different aspects of the problem or alternative solution strategies. Each team will report weekly through a short blog post and brief presentation of their progress. Responsibility for these tasks will rotate from week to week within each team.