Technische Universität München Robotics and Embedded Systems

Machine Learning

Lecturer Prof. Dr. Jürgen Schmidhuber
Module Modul IN2064
Type Lecture
Language English
Semester WS 2008/2009
ECTS 6.0
Audience Elective course for students of Informatics (Diplom 5+, Bachelor 5+, Master 1+)
Elective course for students of Business Informatics (Bachelor 5+)
Time & Location Thu 10:00 - 12:00 MI 00.13.009A
Fri 08:30 - 10:00 MI 00.13.009A
Certificate Based on written examination. The exam will take place on Tue, 10.2.2009, 10:00, room 03.07.023


UPDATED The lecture is over.

Thu. we will have one 45min lecture at 10:00 sharp. The tutorial will take place immediately afterwards, i.e. 10:45-12:00.

The Fri lecture will start at 08:30 sharp.

The plan is have regular lectures throughout the semester, except on the obvious holidays. The lecture started on Thu, 16 Oct. Last lecture will be on Fri, 6 Feb 2009.


The tutorial consists of (mostly) weekly worksheets to be completed by the student, and discussion of the exercises. Some of the tasks may involve programming, but this is not a programming style tutorial - the focus lies on implementing ML algorithms. The worksheets will be posted here.


This lecture will take you on a journey through the exciting and highly active field of Machine Learning, which has applications in areas as diverse as web searches, robotics, data mining, environmental sciences, medical data analysis, and many more. The first part of the lecture loosely follows the textbook by Chris Bishop, referenced below, and uses a lot of his material. You are highly recommended to have a look into it for answers to fundamental questions, and for more in-depth information. Here is an overview of the topics covered (at least cursorily) by the lecture, where arrows indicate our flow of argument rather than historical derivation:


We will start with laying the "groundwork" of ancient statistical techniques such as Bayes classifiers and Linear Discriminant Analysis as well as probabilistic regression. In this context, it would help to be somewhat familiar with the contents of Analysis I/II, Linear Algebra I/II and Probability Theory. The basic principles of kernel methods when combined with our probabilistic framework will lead us the very successful class of Support Vector Machines. It will also allow us to understand other well-established methods such as feed forward neural networks and recurrent relatives. Besides being the basis for neural network training, our discussion of gradient descent methods will also branch off to evolutionary algorithms and their modern descendants. Another important algorithm goes by the name of Expectation Maximization, and leads us to Hidden Markov Models (HMMs) and the Kalman filter, alternatives to recurrent networks when it comes to time series processing and dynamical systems prediction. Eventually, we will give a brief introduction to Reinforcement Learning techniques.



We started a Google group for this lecture. The group is invitation only, so if you want to be added send an email to Christian.

Suggested Reading

[1] David J. C. MacKay. Information theory, inference, and learning algorithms. Cambridge Univ. Press, 2008.
[2] Toby Segaran. Programming collective intelligence. O'Reilly, Beijing, 2007.
[3] Karsten Weicker. Evolutionäre Algorithmen. Teubner, 2. edition, 2007.
[4] Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, Berlin, New York, 2006.
[5] Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern classification. Wiley, New York, 2001.
[6] Tom M. Mitchell. Machine learning. McGraw-Hill, Boston, Mass., 1997.
[7] V. N. Vapnik. The nature of statistical learning theory. Springer-Verlag New York, Inc., New York, NY, USA, 1995.