Machine Learning

General information

Objectives

Prerequisites

Content

Introduction to machine learning: designing a machine learning system, learning settings and tasks, decision trees, k-nearest-neighbour estimation. Mathematical foundations: linear algebra, probability theory, statistical tests. Bayesian decision theory, maximum likelihood and Bayesian parameter estimation. Probabilistic graphical models, inference, parameters and structure learning. Clustering: k-means, hierarchical clustering. Kernel Machines: kernels, reproducing kernel Hilbert spaces, representer theorem, support vector machines for classification, regression and ranking, kernel construction, kernels for structured data. Statistical Learning Theory: PAC learning, consistency, VC dimension, generalization and models comparison. Applications to text categorization and bioinformatics. Course Information Instructor: Andrea Passerini Email: Office hours: Tuesday 10:30-12:30 Lecture time and place: Tuesday 8:30-10:30 Friday 10:30-12:30 Bibliography: R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification (2nd edition), Wiley-Interscience, 2001. C.M.Bishop, Pattern Recognition and Machine Learning, Springer, 2006. J.Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004. B. Schölkopf and A.J. Smola,Learning with Kernels, The MIT Press, 2002. T. Mitchell, Machine Learning, McGraw Hill, 1997. Material: Slides and handouts (pdf format) Introduction [slides] [handouts] Decision Trees [slides] [handouts] K-nearest neighbours [slides] [handouts] Linear algebra [slides] [handouts] Probability theory [slides] [handouts] Bayesian decision theory [slides] [handouts] Parameter estimation [slides] [handouts] Bayesian Networks [slides] [handouts] Inference in BN [slides] [handouts] Learning BN [slides] [handouts] Naive Bayes [slides] [handouts] Expectation Maximization [slides] [handouts] BN software [software] [data] [example] Linear discriminant functions [slides] [handouts] Support Vector Machines [slides] [handouts] Non-linear Support Vector Machines [slides] [handouts] Kernel Machines [slides] [handouts] Kernels [slides] [handouts] Evaluation [slides] [handouts] Clustering [slides] [handouts] Exams Modality: Project [description] and oral examination Projects: (1) Hierarchical gene clustering [desc] [data] (2) Patient classification by gene expression [desc] [data] (3) Bayesian Network of pathologies [desc] [data] (4) Protein subcellular localization [desc] [data] (5) Disulphide bonding state prediction [desc] [data] (6) Comparing learning algorithms [desc] [data]