Written by Andrew Moore, the document is available as PDF.
The Decision Tree is one of the most popular classification algorithms in current use in Data Mining and Machine Learning. This tutorial can be used as a self-contained introduction to the flavour and terminology of data mining without needing to review many statistical or probabilistic pre-requisites. If you're new to data mining you'll enjoy it, but your eyebrows will raise at how simple it all is! After having defined the job of classification, we explain how information gain (next Andrew Tutorial) can be used to find predictive input attributes. We show how applying this procedure recursively allows us to build a decision tree to predict future events. We then look carefully at a question so fundamental, it is the basis for much of all statistics and machine learning theory: how do you choose between a complicated model t
data mining
Bayesian Networks Tutorial
Tutorial slides by Andrew Moore available as PDF.
The tutorial first reviews the fundamentals of probability (but to do that properly, please see the earlier Andrew lectures on Probability for Data Mining). It then discusses the use of Joint Distributions for representing and reasoning about uncertain knowledge. Having discussed the obvious drawback (the curse of dimensionality) for Joint Distributions as a general tool, we visit the world of clever tricks involving indepedence and conditional independence that allow us to express our uncertain knowledge much more succinctly. And then we beam with pleasure as we realize we've got most of the knowledge we need to understand and appreciate Bayesian Networks already. The remainder of the tutorial introduces the important question of how to do inference with Bayesian Networks.
The tutorial first reviews the fundamentals of probability (but to do that properly, please see the earlier Andrew lectures on Probability for Data Mining). It then discusses the use of Joint Distributions for representing and reasoning about uncertain knowledge. Having discussed the obvious drawback (the curse of dimensionality) for Joint Distributions as a general tool, we visit the world of clever tricks involving indepedence and conditional independence that allow us to express our uncertain knowledge much more succinctly. And then we beam with pleasure as we realize we've got most of the knowledge we need to understand and appreciate Bayesian Networks already. The remainder of the tutorial introduces the important question of how to do inference with Bayesian Networks.
Category: tutorial from http://www.autonlab.org
Support Vector Machines Tutorial
Tutorial slides by Andrew Moore available as PDF.
We review the idea of the margin of a classifier, and why that may be a good criterion for measuring a classifier's desirability. Then we consider the computational problem of finding the largest margin linear classifier. At this point we look at our toes with embarrassment and note that we have only done work applicable to noise-free data. But we cheer up and show how to create a noise resistant classifier, and then a non-linear classifier. We then look under a microscope at the two things SVMs are renowned for---the computational ability to survive projecting data into a trillion dimensions and the statistical ability to survive what at first sight looks like a classic overfitting trap.
We review the idea of the margin of a classifier, and why that may be a good criterion for measuring a classifier's desirability. Then we consider the computational problem of finding the largest margin linear classifier. At this point we look at our toes with embarrassment and note that we have only done work applicable to noise-free data. But we cheer up and show how to create a noise resistant classifier, and then a non-linear classifier. We then look under a microscope at the two things SVMs are renowned for---the computational ability to survive projecting data into a trillion dimensions and the statistical ability to survive what at first sight looks like a classic overfitting trap.
Category: tutorial from http://www.autonlab.org
Markov Decision Processes Tutorial
Tutorial slides by Andrew Moore available as PDF.rnrnHow do you plan efficiently if the results of your actions are uncertain? There is some remarkably good news, and some some significant computational hardship. We begin by discussing Markov Systems (which have no actions) and the notion of Markov Systems with Rewards. We then motivate and explain the idea of infinite horizon discounted future rewards. And then we look at two competing approaches to deal with the following computational problem: given a Markov System with Rewards, compute the expected long-term discounted rewards. The two methods, which usually sit at opposite corners of the ring and snarl at each other, are straight linear algebra and dynamic programming. We then make the leap up to Markov Decision Processes, and find that we've already done 82% of the work needed to compute not only the long term rewards of each
Category: tutorial from http://www.autonlab.org
« previous1 next »
