Course Description

Machine learning algorithms are data analysis methods which search data sets for patterns and characteristic structures. Typical tasks are the classification of data, automatic regression and unsupervised model fitting. Machine learning has emerged mainly from computer science and artificial intelligence, and draws on methods from a variety of related subjects including statistics, applied mathematics and more specialized fields, such as pattern recognition and neural computation. Applications are, for example, image and speech analysis, medical imaging, bioinformatics and exploratory data analysis in natural science and engineering:


nonlinear decision boundary linear discriminant analysis gene expression levels
Non-linear decision boundary of a trained support vector machine (SVM) using a radial-basis function kernel. Fisher's linear discriminant analysis (LDA) of four different auditory scenes: speech, speech in noise, noise and music. Gene expression levels obtained from a micro-array experiment, used in gene function prediction.


This course is intended as an introduction to machine learning. It will review the necessary statistical preliminaries and provide an overview of commonly used machine learning methods. Further and more advanced topics will be discussed in the course Statistical Learning Theory, held in the spring semester by Prof. Buhmann.


Other related courses offered at the D-INFK include: Computational Intelligence Lab, Probabilistic Artificial Intelligence, Advanced Topics in Machine Learning, Information Retrieval, Data Mining.

News

Date What?
19.02.
  1. The project grades are now available Link (.xlsx)
18.12.
  1. The Q+A will take place on January 19th in ETF C1 from 10-12am.
24.09
  1. The videos of the lecture are available here: Link
17.09
  1. We got a (larger) room for the lecture on Tuesday: New Schedule
22.08
  1. The first lecture will take place on Tuesday 15.09
  2. The tutorial classes will start as of the second week of the semester. Please note that all five sessions are identical.

Syllabus

Calendar Week Lecture Topics Lecture Slides Tutorial Slides Exercise Sheets & Solutions References (partial list)
38 Introduction to Machine Learning / Course Description ml15_introduction.pdf
39 Taxonomy of Data / Regression ml15_lecture_02.pdf ml_tutorial_01.pdf series1.pdf
solution1.pdf
40 Regression / Bias-Variance ml15_lecture_03.pdf Matlab/Python Intro (.zip)
41 Density Estimation in Regression ml15_lecture_04.pdf ml_tutorial_03.pdf project1_description.pdf
report.tex
Winner Report
42 Gaussian Processes ml15_lecture_05.pdf ml_tutorial_04.pdf series2.pdf
GP Trajectories Visualization
solution2.pdf
43 Numerical Estimation Techniques and Classification ml15_lecture_06.pdf
ml15_lecture_07.pdf
ml_tutorial_05.pdf
43 Linear Discriminant Functions ml15_lecture_08.pdf ml_tutorial_06.pdf project2_description.pdf
report.tex
raw_data.zip
Winner Report
44 Support Vector Machines ml15_lecture_09.pdf ml_tutorial_07.pdf
series3.pdf
solution3.pdf
45 Non-Linear SVMs ml15_lecture_10.pdf ml_tutorial_08.pdf
46 Ensemble Methods ml15_lecture_11.pdf ml_tutorial_09.pdf series4.pdf
solution4.pdf
overfitting_nonlin_SVMs.m
Matlab Skeleton
Matlab Graph
46 Neural Network ml15_lecture_12.pdf ml_tutorial_10.pdf project3_description.pdf
report_template_project3.tex
hand_out.zip
Winner Report
47 Unsupervised Learning: Nonparametric Density Estimation Expert Talk - Deep Learning
ml15_lecture_13.pdf
ml_tutorial_11.pdf
ml_tutorial_11_interactive.pdf
ml_tutorial_11_neural_code.zip
(demo pdf and zip updated)
48 Mixture models ml15_lecture_14.pdf ml_tutorial_12.pdf series5.pdf
solution5.pdf
49 Dimension Reduction ml15_lecture_15.pdf ml_tutorial_13.pdf

Some of the material can only be accessed with a valid nethz account.

General Information

VVZ Information: See here.

Time and Place

Lectures
Mon 14-15 ETF C 1
Tue 08-10 ETF E 1
Tutorials
TimeRoomLast Name
Tue 10-12 CAB G 51 A-B
Wed 13-15 CAB G 11 C-K
Wed 15-17 CAB G 61 L-R
Fri 08-10 CAB G 52 S
Fri 13-15 CAB G 61 T-Z

* All tutorial sessions are identical, please attend the session assigned to you based on the first letter of your last name

Exercises

The exercise problems will contain theoretical Pen&Paper assignments. Please note that it is not mandatory to submit solutions, a Testat is not required in order to participate in the exam. We will publish exercise solutions after one week.

If you choose to submit:

Project

Part of the coursework will be a project, carried out in groups of 3 students. The goal of this project is to get hands-on experience in machine learning tasks. The project grade will constitute 30% of the total grade. More details on the project will be given in the tutorials.

Exam

The mode of examination is written, 120 minutes length. The language of examination is English. As written aids, you can bring two A4 pages (i.e. one A4 sheet of paper), either handwritten or 11 point minimum font size. The written exam will constitute at least 70% of the total grade.

Resources

Text Books

C. Bishop. Pattern Recognition and Machine Learning. Springer 2007.
This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. Available from ETH-HDB and ETH-INFK libraries.

R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001.
The classic introduction to the field. An early edition is available online for students attending this class, the second edition is available from ETH-BIB and ETH-INFK libraries.

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
Another comprehensive text, written by three Stanford statisticians. Covers additive models and boosting in great detail. Available from ETH-BIB and ETH-INFK libraries.
A free PDF version (second edition) is available online

L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
This book is a compact treatment of statistics that facilitates a deeper understanding of machine learning methods. Available from ETH-BIB and ETH-INFK libraries.

D. Barber. Bayesian Reasoning and Machine Learning. Cambridge University Press, 2012.
This book is a compact and extensive treatment of most topics. Available for personal use online: Link.

K. Murphy. Machine Learning: A Probabilistic Perspective. MIT, 2012.
Unified probabilistic introduction to machine learning. Available from ETH-BIB and ETH-INFK libraries.

S. Shalev-Shwartz, and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
This recent book covers the mathematical foundations of machine learning. Available for personal use online: Link.

Matlab

The official Matlab documentation is available online at the Mathworks website (also in printable form). If you have trouble accessing Matlab's built-in help function, you can use the online function reference on that page or use the command-line version (type help <function> at the prompt). There are several primers and tutorials on the web, a later edition of this one became the book Matlab Primer by T. Davis and K. Sigmon, CRC Press, 2005.

Discussion Forum

For discussions with other students and to test your understanding, we encourage you to ask and reply questions regarding the Machine Learning Lecture on the VIS inforum. Use it to ask questions of general interest and interact with other students of this class. We regularly visit the board to provide answers.

Previous Exam

Contact

Instructor: Prof. Joachim M. Buhmann
Head Assistant: Stefan Bauer
Assistants: Gabriel Krummenacher, Alexey Gronskiy, Viktor Wegmayr, Nico Gorbach, Jie Song, Julian Viereck, Yatao Bian, David Tedaldi, Veselin Raychev, Antonio Loquercio, Besmira Nushi, Emiliano Diaz Salas Porras