Course Description

Machine learning algorithms are data analysis methods which search data sets for patterns and characteristic structures. Typical tasks are the classification of data, automatic regression and unsupervised model fitting. Machine learning has emerged mainly from computer science and artificial intelligence, and draws on methods from a variety of related subjects including statistics, applied mathematics and more specialized fields, such as pattern recognition and neural computation. Applications are, for example, image and speech analysis, medical imaging, bioinformatics and exploratory data analysis in natural science and engineering:

nonlinear decision boundary linear discriminant analysis gene expression levels
Non-linear decision boundary of a trained support vector machine (SVM) using a radial-basis function kernel. Fisher's linear discriminant analysis (LDA) of four different auditory scenes: speech, speech in noise, noise and music. Gene expression levels obtained from a micro-array experiment, used in gene function prediction.

This course is intended as an introduction to machine learning. It will review the necessary statistical preliminaries and provide an overview of commonly used machine learning methods. Further and more advanced topics will be discussed in the course Statistical Learning Theory, held in the spring semester by Prof. Buhmann.


Date What?
01.01.2013 The ML Q&A session will be held on Monday February 4'th 2013 between 16:00-18:00 at CAB G.61.
Please send your questions until Monday morning (Feb 4) to
  1. The exam of 2011 is now available. This week's tutorial will be dedicated to solving it.
  2. The exam will take place on February 6, 2013, between 9-11
  3. There are no tutorial classes next week (19-21/12). Instead we will conduct a Q&A session few days before the exam. A separate announcement with the specific details will be sent by email, as well as on the webpage.
08.11 Please note that lecture8 was updated.
29.10 Please note that the slides of lecture6 were updated and now include Fisher’s linear discriminant analysis.
16.10 Please note that lecture5 was updated.
02.10 The topic of the tutorial this week is "efficient Matlab programming". If possible, please bring a laptop with Matlab already installed.
  1. The first lecture will take place on Tuesday 18.09
  2. The tutorial classes will start as of the second week of the semester. Please note that all three sessions are identical.


Calendar Week Lecture Topics Lecture Slides Tutorial Slides Exercise Sheets & Solutions References (partial list)
38 Introduction to Machine Learning information.pdf
39 Introduction: Supervised and Unsupervised Learning ml12_lecture_02.pdf iml_tutorial_01.pdf series1.pdf
40 Supervised Learning ml12_lecture_03.pdf iml_tutorial_02.pdf
41 Parametric Models ml12_lecture_04.pdf iml_tutorial_03.pdf series2.pdf
Rao Cramer: Cover&Thomas 11.10
42 Numerical Estimation Techniques ml12_lecture_05.pdf iml_tutorial_04.pdf Ripley: pp. 72-75
43 Linear Discriminant Functions ml12_lecture_06.pdf iml_tutorial_05Part1.pdf iml_tutorial_05Part2.pdf series3.pdf
44 Support Vector Machines ml12_lecture_07.pdf iml_tutorial_06.pdf Christianini, Shawe-Taylor: pp. 79-112
45 Non-Linear SVMs ml12_lecture_08.pdf iml_tutorial_07.pdf series4.pdf
46 Ensemble Methods: Boosting, Bagging ml12_lecture_09.pdf
47 Regression: Least Squares, Ridge, Lasso ml12_lecture_10.pdf
iml_tutorial_09.pdf series5.pdf
48 Unsupervised Learning: Nonparametric Density Estimation ml12_lecture_11.pdf iml_tutorial_10Part1.pdf iml_tutorial_10Part2.pdf
49 Clustering: K-means. Mixture models
EM algorithm
ml12_lecture_12.pdf iml_tutorial_11Part1.pdf iml_tutorial_11Part2.pdf series6.pdf
50 Time Series: Stochastic processes, Markov models, Hmm ml12_lecture_13.pdf
51 Dimension reduction ml12_lecture_14.pdf series7.pdf

Some of the material is password protected, send an email to to obtain it.

General Information

Time and Place

Lectures Mon 14-15 CAB G61
Tue 10-12 HG D 1.1
Tutorials Wed 15-17 CAB G61
Thu 15-17 ML F34
Fri 08-10 CAB G52

* All tutorial sessions are identical, please only attend one session.


The exercise problems will include theoretical and programming problems. All programming will be done in Matlab. Please note that it is not mandatory to submit solutions, a Testat is not required in order to participate in the exam. We will publish exercise solutions after one week.

If you choose to submit:

Performance Assessment


The mode of examination is written, 120 minutes length. The language of examination is English. As written aids, you can bring two A4 pages (i.e. one A4 sheet of paper), either handwritten or 11 point minimum font size.


Text Books

C. Bishop. Pattern Recognition and Machine Learning. Springer 2007.
This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. Available from ETH-HDB and ETH-INFK libraries.

R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001.
The classic introduction to the field. An early edition is available online for students attending this class, the second edition is available from ETH-BIB and ETH-INFK libraries.

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
Another comprehensive text, written by three Stanford statisticians. Covers additive models and boosting in great detail. Available from ETH-BIB and ETH-INFK libraries.
A free PDF version (second edition) is available online

L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
This book is a compact treatment of statistics that facilitates a deeper understanding of machine learning methods. Available from ETH-BIB and ETH-INFK libraries.


The official Matlab documentation is available online at the Mathworks website (also in printable form). If you have trouble accessing Matlab's built-in help function, you can use the online function reference on that page or use the command-line version (type help <function> at the prompt). There are several primers and tutorials on the web, a later edition of this one became the book Matlab Primer by T. Davis and K. Sigmon, CRC Press, 2005.

Discussion Forum

We maintain a discussion board at the VIS inforum. Use it to ask questions of general interest and interact with other students of this class. We regularly visit the board to provide answers.

Previous Exams

Previous Year's Slides

You can download the complete set of last year's (2010) slides here:
iml_2010_complete_set.pdf, 4 slides per page: iml_2010_complete_set-4up.pdf
Please note that this year's material and/or slides may change.


You can access the Java applets Prof. Buhmann uses in the lecture (plus some others) here.


Instructor: Prof. J. M. Buhmann
Head Assistant: Sharon Wulff
Assistants: Alberto Giovanni Busetto, Morteza Chehreghani, Alexey Gronskiy, Gabriel Krummenacher, Dmitry Laptev, Dr. Dwarikanath Mahapatra

Valid XHTML 1.0 Transitional