Course Description

Machine learning algorithms are data analysis methods which search data sets for patterns and characteristic structures. Typical tasks are the classification of data, automatic regression and unsupervised model fitting. Machine learning has emerged mainly from computer science and artificial intelligence, and draws on methods from a variety of related subjects including statistics, applied mathematics and more specialized fields, such as pattern recognition and neural computation. Applications are, for example, image and speech analysis, medical imaging, bioinformatics and exploratory data analysis in natural science and engineering:


nonlinear decision boundary linear discriminant analysis gene expression levels
Non-linear decision boundary of a trained support vector machine (SVM) using a radial-basis function kernel. Fisher's linear discriminant analysis (LDA) of four different auditory scenes: speech, speech in noise, noise and music. Gene expression levels obtained from a micro-array experiment, used in gene function prediction.

Other related courses offered at the D-INFK include: Computational Intelligence Lab, Probabilistic Artificial Intelligence, Advanced Topics in Machine Learning, Information Retrieval, Data Mining.

News

Date What?
13.01
  1. Solution to Series 4 Code updated (THX for feedback, see Piazza)
10.01
  1. Solution to Series 4 updated (THX for feedback, see Piazza)
  2. Tutorial on neural nets updated (THX for feedback, see Piazza)
04.01
  1. Typo corrected: the exam is 180 minutes, not 120.
23.12
  1. Tutorial 11 (Neural Nets) updated because of wrong layout
  2. Solution for series 6 added
20.12
  1. Tutorial 12 (presentation + code) added
  2. Solution 5 added
19.12
  1. Lecture 15 (time series) added.
16.12
  1. Tutorial 11 added
  2. Series 6 added
12.12
  1. Lecture 14 (mixture models) added.
05.12
  1. Lecture 13 added
  2. Series 5 added
02.12
  1. Tutorial 10 (Mixture Models) and code for it added
  2. Solution 4 added
30.11
  1. Series 3 and Solution for Series 3 are updated (typos)
28.11
  1. Lecture 12 added.
27.11
  1. Added matlab skeleton for series 4. I'm sorry for inconvenience.
20.11
  1. Solution to series 3 added (code TBA)
  2. Lecture 11 (ensemble) added.
  3. Newer exams (2013-2015) added.
18.11
  1. Structured SVM tutorial 08 posted.
  2. Lecture 10 (structured SVM) updated.
13.11
  1. Lecture 10 posted.
08.11
  1. Lecture 9 (SVM) UPDATED (added slides 23+).
  2. Exercise series 3 supplementary material uploaded
07.10
  1. Exercise series 3 added
  2. Solution series 2 aded
06.11
  1. Lecture 9 (SVM) added.
05.10
  1. Lecture 8 (perception) (added slides >23 with topic LDA).
01.11
  1. Lecture 8 UPDATED once more.
  2. Tutorial 6 added.
31.10
  1. Lecture 8 (perception) ADDED and UPDATED (slide 10).
28.10
  1. Lecture 3_GP updated (slide 6, Bayesian Regr Variance Calc)
27.10
  1. Tutorial 5 UPD.
26.10
  1. Solution to series 1 and tutorial 5 added. Sorry for a technical delay!
25.10
  1. Exercise series 2 added.
24.10
  1. Lecture 7 added.
24.10
  1. Lecture 3 updated (typos for SVD); Applets updated
23.10
  1. Tutorial 4 added
23.10
  1. Lecture 6 (as of 24.10.2016) added
17.10
  1. Lecture 4 added
17.10
  1. Lecture 3_GP was updated (typos)
14.10
  1. Tutorial 3 (Param estimation) added
13.10
  1. Tutorial 3 will be added on Friday
10.10
  1. UPDATED Lecture 3_GP (Gaussian Proc) updated
10.10
  1. Lecture 3_GP (Gaussian Proc) added
30.09
  1. Video lecture recordings are available at the ETH Video Portal
27.09
  1. Lecture 3 added
27.09
  1. CHANGE of room on Tuesdays: lecture in HG E7 and streaming in E3
  2. CHANGE of room on Friday mornings: tutorial 08:00 - 10:00 in ML E12
  3. See the plans in the respective section
26.09
  1. Lecture pdf loaded without animation
  2. Added the Intro on Projects
26.09
  1. Lecture 2 (Monday, 26th Sept) has been added.
20.09
  1. There are four exercise time slots each week. The assignment of participants to exercise sessions will follow your LAST NAMEs. Below you find the initial letter of your last names as well as the exercise session that you are assigned to.
    A-F Wednesday (13:00-15:00)
    G-K Wednesday (15:00-17:00)
    L-R Friday (08:00-10:00)
    S-Z Friday (13:00-15:00)
  2. To account for the scale of this course, we will answer questions regarding lectures exercises and projects on Piazza. To allow for optimal information flow, please ask your content-related questions on this platform (rather than via direct emails) and label them accordingly (e.g., by indicating which lecture / project your question refers to). In this manner, your question and our answer are visible to everyone. Consequently, please read existing question-answer pairs before asking new questions. You can enroll to our Piazza course page via the Piazza link
19.09
  1. The first lecture will take place on Tuesday 20.09
  2. The tutorial classes will start as of the second week of the semester. Please note that all four sessions are identical.

Syllabus

Calendar Week Lecture Topics Lecture Slides Tutorial Slides Exercise Sheets & Solutions References (partial list)
38 Organization / Introduction ml16_course_information.pdf
ml16_lecture_01.pdf
39 Data types/Regression/Bias-Variance ml16_lecture_02.pdf
ml16_lecture_03.pdf [UPD]		 ml16_tutorial_01.pdf intro projects.pdf
40 Regression (cont.) / GP ml16_lecture_03.pdf (cont.) ml16_tutorial_02.pdf ml16_exercise_01.pdf
solution1_week2.pdf
41 GP ml16_lecture_03_gp.pdf [UPDATED] ml16_tutorial_03.pdf
42 Density estimation
Max Likelihood		 ml16_lecture_04.pdf ml16_tutorial_04.pdf ml16_exercise_02.pdf
solution2_week4.pdf
43 Cross-validation ml16_lecture_06.pdf
ml16_lecture_07.pdf Appplets:
Classifier
Cluster
Density
HMM		 ml16_tutorial_05_handout.pdf [UPD] For applets, use Safari and add the "https://ml2.inf.ethz.ch" to white list
44 Perception ml16_lecture_08.pdf [UPD 05.11, +LDA] ml16_tutorial_06.pdf [UPD 07.11, LDA] ml16_exercise_03.pdf [UPD typos]
solution3_week6.pdf [adding details]		 Suppl. material for Series 3:
week6_problem3.py a9a
45 SVM ml16_lecture_09.pdf [UPD 08.11, slides 23+] ml16_tutorial_07_presentation.pdf
ml16_tutorial_07_handout.pdf
ml16_tutorial_07_python.ipynb (iPython demo)
46 Structured SVM ml16_lecture_10.pdf [UPD 18.11, +missing slides] ml16_tutorial_08.pdf ml16_exercise_04.pdf
ml16_exercise_04_matlab_skeleton.zip
solution4_week8.pdf
ml16_exercise_04_matlab_solution.zip
47 Ensemble Methods ml16_lecture_11.pdf
48 Neural ml16_lecture_12.pdf ml16_tutorial_10_presentation.pdf
ml16_tutorial_10_code.zip		 ml16_exercise_05.pdf
solution5_week10.pdf
49 Unsupervised Learning ml16_lecture_13.pdf ml16_tutorial_11.pdf
50 Mixture Models ml16_lecture_14.pdf [UPD 19.12] ml16_tutorial_12.pdf
ml16_tutorial_12_code.zip		 ml16_exercise_06.pdf
solution6_week11.pdf
51 Time Series ml16_lecture_15.pdf

Some of the material can only be accessed with a valid nethz account.

General Information

VVZ Information: See here.

Time and Place

Lectures
Mon 14-15 ETF C 1
Tue 08-10 (NEW) HG E 7 (lecture)
(NEW) HG E 3 (live streaming)
Tutorials
TimeRoomRemarks
Wed 13-15 CAB G 11 Surnames A-F
Wed 15-17 CAB G 61 Surnames G-K
Fri 08-10 (NEW) ML E 12 Surnames L-R
Fri 13-15 CAB G 61 Surnames S-Z

* All tutorial sessions are identical, please attend the session assigned to you based on the first letter of your last name

Exercises

The exercise problems will contain theoretical Pen&Paper assignments. Please note that it is not mandatory to submit solutions, a Testat is not required in order to participate in the exam. We will publish exercise solutions after one week.

If you choose to submit:

Project

Part of the coursework will be a project, carried out in groups of 3 students. The goal of this project is to get hands-on experience in machine learning tasks. The project grade will constitute 30% of the total grade. More details on the project will be given in the tutorials.

Exam

The mode of examination is written, 180 minutes length. The language of examination is English. As written aids, you can bring two A4 pages (i.e. one A4 sheet of paper), either handwritten or 11 point minimum font size. The written exam will constitute at 70% of the total grade.

Questions and Discussion Forum

To account for the scale of this course, we will answer questions regarding lectures exercises and projects on Piazza. To allow for optimal information flow, please ask your content-related questions on this platform (rather than via direct emails) and label them accordingly (e.g., by indicating which lecture / project your question refers to). In this manner, your question and our answer are visible to everyone. Consequently, please read existing question-answer pairs before asking new questions.

You can enroll to our Piazza course page via the Piazza link

Text Books

C. Bishop. Pattern Recognition and Machine Learning. Springer 2007.
This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions. Available from ETH-HDB and ETH-INFK libraries.

R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, second edition, 2001.
The classic introduction to the field. An early edition is available online for students attending this class, the second edition is available from ETH-BIB and ETH-INFK libraries.

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2001.
Another comprehensive text, written by three Stanford statisticians. Covers additive models and boosting in great detail. Available from ETH-BIB and ETH-INFK libraries.
A free PDF version (second edition) is available online

L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2004.
This book is a compact treatment of statistics that facilitates a deeper understanding of machine learning methods. Available from ETH-BIB and ETH-INFK libraries.

D. Barber. Bayesian Reasoning and Machine Learning. Cambridge University Press, 2012.
This book is a compact and extensive treatment of most topics. Available for personal use online: Link.

K. Murphy. Machine Learning: A Probabilistic Perspective. MIT, 2012.
Unified probabilistic introduction to machine learning. Available from ETH-BIB and ETH-INFK libraries.

S. Shalev-Shwartz, and S. Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
This recent book covers the mathematical foundations of machine learning. Available for personal use online: Link.

Matlab

The official Matlab documentation is available online at the Mathworks website (also in printable form). If you have trouble accessing Matlab's built-in help function, you can use the online function reference on that page or use the command-line version (type help <function> at the prompt). There are several primers and tutorials on the web, a later edition of this one became the book Matlab Primer by T. Davis and K. Sigmon, CRC Press, 2005.

Previous Exam

Contact

(For questions regarding the course, please use the Piazza system, not the e-mails: Questions)

Instructor: Prof. Joachim M. Buhmann
Head Assistant: Carsten Eickhoff
Assistants: To be populated