Statistical Learning Theory, Spring Semester 2016

Instructor

Prof. Dr. J.M. Buhmann

Assistants

Viktor Wegmayr
Alexey Gronskiy (contact TA)

General Information

The ETHZ Course Catalogue information can be found here.

The course covers advanced methods of statistical learning. The fundamentals of Machine Learning as presented in the course "Introduction to Machine Learning" are expanded and, in particular, the following topics are discussed:

  • Statistical Learning Theory: How can we measure the quality of a classifier? Can we give any guarantees for the prediction error?
  • Variational Methods and Optimization: We consider optimization approaches for problems where the optimizer is a probability distribution. Concepts we will discuss in this context include:
    • Maximum Entropy
    • Information Bottleneck
    • Deterministic Annealing
  • Clustering: The problem of sorting data into groups without using training samples. This requires a definition of ``similarity'' between data points and adequate optimization procedures;
  • Model Selection: We have already discussed how to fit a model to a data set in ML I, which usually involved adjusting model parameters for a given type of model. Model selection refers to the question of how complex the chosen model should be. As we already know, simple and complex models both have advantages and drawbacks alike;
  • Reinforcement Learning: The problem of learning through interaction with an environment which changes. To achieve optimal behavior, we have to base decisions not only on the current state of the environment, but also on how we expect it to develop in the future;:

Time and Place

Type Time Place
Lectures Mon 14-16 HG G 5
Exercises Mon 16-17 HG G 5

Student Forum

The link (member approval with google e-mail would be needed). Please feel free to use it for any questions, comments to the TA's, for sharing ideas and discussing assignments, projects and anything related to SLT with other students.

News

Date Reason
June 19 Typos corrected in the tutorial on Mean-Field approximation (Variational Inference).
May 9 Project report template (which is in fact a NIPS paper one) is available.
May 9 Here's the link (member approval with google e-mail would be needed) to the Student Forum SLT. Please feel free and creative to use it for sharing ideas on home assignments, projects, etc (it's by no means a competition).
Apr 28 PROJECTS: Dear all, we've made the project descriptions available in the respective section below. Please find there the general project information (workflow, deadlines, grading etc), as well as and project descriptions.
Apr 20 We started putting solutions online. Please keep track of them.
Apr 19 Preliminary project explanation document is updated! Here (updated) you can find the updated preliminary SLT project gradings.
Apr 18 1. Here (updated) you can find preliminary SLT project gradings. The projects themselves are expected to be delivered on Monday, May 2nd. Or a bit earlier, time premitting.

2. We decided to put an "Sechselauten exercise" so that you could practice. It is an "joker" one in the sense that we will count it towards Testat if you hand it in but if you don't it will not count towards not getting Testat.
Apr 11 We started putting non-handwritten tutorials on the web.
Mar 29 Happy Easter to those who celebrate! We decided to put an "Easter exercise" so that you could practice. It is an "joker" one in the sense that we will count it towards Testat if you hand it in but if you don't it will not count towards not getting Testat.
Mar 15 Corrected some typos in Exercise 3.
Mar 10 Added the coed for tutorial and corrected some typos in Exercise 2.
Mar 02 We aplogize for a delay with the exercise, it will be delivered on 3d of March. For those needing the Testat we will provide grading relaxations due to this delay ;-)
Feb 21 Lecture for Feb 22 added.
Feb 19 Dear all, we decided to start the Tutorials from the first week, i.e. Feb 22.
Feb 18 Time and place added
Jan 11 Creation of the website. Welcome!

Material

Date Lecture/Tutorial Slides Exercise Series, Hometasks
Feb 22 Lecture [pdf]
Feb 29 Lecture [pdf] Exercise [pdf]
Mar 7 Lecture [pdf]
MCMC Tutorial Code [zip]		 Exercise [pdf]
Solution [pdf]
Mar 14 Tutorial Notes [pdf] Exercise [pdf]
Solution [pdf]
Mar 21 Lecture [pdf]
Tutorial Notes [pdf]		 Exercise [pdf]
Solution [pdf]
Mar 28 (Easter) No class "Extra/Joker" Exercise [pdf, handwritten]
(solutions to Viktor;
typed version later,
sorry for the inconvenience)
Apr 4 Lecture [pdf]
Tutorial Notes [pdf]		 Exercise [pdf]
Solution [pdf]
Apr 11 Lecture [pdf]
Tutorial Notes [pdf]		 Exercise [pdf]
Solution [pdf]
Apr 18 (Sechselauten) No class
Prelimiary Project Grading (upd) [pdf]		 "Extra/Joker" Exercise [pdf, handwritten]
(solutions to Viktor;
typed version later,
sorry for the inconvenience)
Apr 25 Lecture [pdf]
Tutorial Notes [pdf, updated]
 Exercise [pdf]
May 2
May 9 Lecture [pdf] Exercise [pdf]
Solution [pdf]
May 16 No class
May 23 Lecture [pdf] No exercise
May 30 Lecture [pdf] Project presentations

Projects

Diffusion MRI bundle segmentation Semantic clustering of nouns
Project description [pdf] Project description [pdf]

Reading

Web Acknowledgements

The web-page code is based (with modifications) on the one of the course on Machine Learning (Fall Semester 2013; Prof. A. Krause).