Course Description

Machine learning algorithms are data analysis methods which search data sets for patterns and characteristic structures. Typical tasks are the classification of data, automatic regression and unsupervised model fitting. Machine learning has emerged mainly from computer science and artificial intelligence, and draws on methods from a variety of related subjects including statistics, applied mathematics and more specialized fields, such as pattern recognition and neural computation. Applications are, for example, image and speech analysis, medical imaging, bioinformatics and exploratory data analysis in natural science and engineering:


nonlinear decision boundary linear discriminant analysis gene expression levels
Non-linear decision boundary of a trained support vector machine (SVM) using a radial-basis function kernel. Fisher's linear discriminant analysis (LDA) of four different auditory scenes: speech, speech in noise, noise and music. Gene expression levels obtained from a micro-array experiment, used in gene function prediction.


This course is intended as an introduction to machine learning. It will review the necessary statistical preliminaries and provide an overview of commonly used machine learning methods. Further and more advanced topics will be discussed in the course Statistical Learning Theory, held in the spring semester by Prof. Buhmann.


Other related courses offered at the D-INFK include: Computational Intelligence Lab, Probabilistic Graphical Models for Image Analysis, Probabilistic Artificial Intelligence, Advanced Topics in Machine Learning, Information Retrieval, Big Data, Data Mining.

News

Date What?
17.01. As explained in the Q&A, some typos and mistakes were corrected.
  1. Some typos in the solution to exercise 2.5 were corrected: solution2.pdf.
  2. The following mistakes in lecture 13 were corrected: Slide 21: Definition of f_l(s_t), Slide 23: Expression b_l(s_t) in step 6, Slide 32: E_k(b).
24.11.
  1. Slides for lecture 10 (Ensemble Methods) were updated.
04.11.
  1. Added slides for tutorial 5 (the linear classification tutorial is now called iml_tutorial_06.pdf)
02.10.
  1. There are a lot of questions about the project, you can find answers to the most frequent here now: http://project.las.ethz.ch/faq.
24.09
  1. The topic of the tutorial next week is going to be "efficient Matlab programming". If possible, please bring a laptop with Matlab already installed.
  2. You can register your group for the project now: http://project.las.ethz.ch/register.
17.09
  1. We got new (bigger) rooms for the lecture and tutorials: New Schedule
  2. There is a new exercise class on Wed 13-15
  3. The assignment of students to exercise classes was updated to reflect those changes.
22.08
  1. The first lecture will take place on Tuesday 16.09
  2. The tutorial classes will start as of the second week of the semester. Please note that all four sessions are identical.

Syllabus

Calendar Week Lecture Topics Lecture Slides Tutorial Slides Exercise Sheets & Solutions References (partial list)
38 Introduction to Machine Learning / Course Description ml-01-introduction.pdf
39 Taxonomy of Data / Regression ml14_lecture_02.pdf iml_tutorial_01.pdf series1.pdf
solution1.pdf
40 Regression / Bias-Variance ml14_lecture_03.pdf iml_tutorial_02.pdf
41 Numerical Estimation Techniques ml14_lecture_04.pdf iml_tutorial_03.pdf
42 Classification ml14_lecture_05.pdf iml_tutorial_04.pdf series2.pdf
solution2.pdf
43 Parametric Models ml14_lecture_06.pdf iml_tutorial_05.pdf
43 Design of Linear Discriminant Functions ml14_lecture_07.pdf iml_tutorial_06.pdf series3.pdf
solution3.pdf
44 Support Vector Machines ml14_lecture_08.pdf iml_tutorial_07_part1.pdf
iml_tutorial_07_part2.pdf
45 Non-Linear SVMs ml14_lecture_09.pdf iml_tutorial_08.pdf series4.pdf
solution4.pdf
46 Ensemble Methods ml14_lecture_10.pdf iml_tutorial_09.pdf
47 Unsupervised Learning: Nonparametric Density Estimation ml14_lecture_11.pdf
ml14_lecture_11_NeuralNetworks.pdf		 iml_tutorial_10.pdf series5.pdf
solution5.pdf
48 Mixture models ml14_lecture_12.pdf iml_tutorial_11.pdf
49 Time Series: Stochastic processes, Markov models, Hmm ml14_lecture_13.pdf iml_tutorial_12.pdf series6.pdf
solution6.pdf
50 Dimension reduction ml14_lecture_14.pdf

Some of the material can only be accessed with a valid nethz account.

General Information

VVZ Information: See here.

Time and Place

Lectures
Mon 14-15 ETF E1
Tue 10-12 NO C60
Tutorials
TimeRoomLast Name
Wed 13-15 CAB G11 A-G
Wed 15-17 CAB G61 H-N
Thu 15-17 CAB G59 O-R
Fri 08-10 CHN E46 S
Fri 13-15 ML E12 T-Z

* All tutorial sessions are identical, please attend the session assigned to you based on the first letter of your last name

Exercises

The exercise problems will contain theoretical Pen&Paper assignments. Please note that it is not mandatory to submit solutions, a Testat is not required in order to participate in the exam. We will publish exercise solutions after one week.

If you choose to submit:

Project

Part of the coursework will be a project, carried out in groups of 3 students. The goal of this project is to get hands-on experience in machine learning tasks. The project grade will constitute 30% of the total grade. More details on the project will be given in the tutorials.

Exam

The mode of examination is written, 120 minutes length. The language of examination is English. As written aids, you can bring two A4 pages (i.e. one A4 sheet of paper), either handwritten or 11 point minimum font size. The written exam will constitute 70% of the total grade.

Resources

Text Books

C. Bishop. Pattern Recognition and Machine Learning. Springer 2007.
This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. Available from ETH-HDB and ETH-INFK libraries.

R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001.
The classic introduction to the field. An early edition is available online for students attending this class, the second edition is available from ETH-BIB and ETH-INFK libraries.

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
Another comprehensive text, written by three Stanford statisticians. Covers additive models and boosting in great detail. Available from ETH-BIB and ETH-INFK libraries.
A free PDF version (second edition) is available online

L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
This book is a compact treatment of statistics that facilitates a deeper understanding of machine learning methods. Available from ETH-BIB and ETH-INFK libraries.

Matlab

The official Matlab documentation is available online at the Mathworks website (also in printable form). If you have trouble accessing Matlab's built-in help function, you can use the online function reference on that page or use the command-line version (type help <function> at the prompt). There are several primers and tutorials on the web, a later edition of this one became the book Matlab Primer by T. Davis and K. Sigmon, CRC Press, 2005.

Discussion Forum

We maintain a discussion board at the VIS inforum. Use it to ask questions of general interest and interact with other students of this class. We regularly visit the board to provide answers.

Previous Exams

Contact

Instructor: Prof. Joachim M. Buhmann
Head Assistant: Gabriel Krummenacher
Assistants: Alexey Gronskiy, Dmitry Laptev, Dr. Dwarikanath Mahapatra, Nico Gorbach, Kate Lomakina, Pratanu Roy, Yatao Bian, Mario Lučić, Darko Makreshanski, Veselin Raychev, Chen Chen