Data Mining

Reading Guide

• Chapter 15 (Random Forests) from Elements of Statistical Learning
• Parts of this (optional) reading are advanced, and so it is recommended for graduate students and ambitious undergrads.
• From the beginning up to and including Section 15.3 is recommended.
• In Section 15.2, there are experiments involving boosting (which we won't cover until much later in the course) and some advanced statistical discussion. Even so, this section is worthwhile to read.
• Note that Section 15.2 mentions a "random forests website"; here is an updated link to that webpage. The page is very interesting and was created by Leo Breiman (an absolute foundational figure within machine learning and statistics) and Adele Cutler.
• In Section 15.3 is very useful. I want to highlight the rules of thumb given just before Section 15.3.1 and also the idea of "Out of Bag Samples" (Section 15.3.1).
• Chapter 13 of Murphy
• This reading is optional but recommended as it offers a modern treat of deep neural networks.
• At the time we cover neural networks in this course, this chapter will be too advanced for almost everyone. I therefore recommend trying to read this chapter later on in the course, after we have covered Logistic Regression (which will be after the midterm)
• That said, you might at least use this chapter as a reference when running experiments with neural networks for either an assignment or the group project.
• SVM tutorial
• Definitely read pages 1 through 3, until just before "Running Example: Splice Site Recognition". Also, definitely read from page 5 from "Large Margin Separation" until page 12 (until just before "Kernels for Sequences")
• For grad students, I recommend also reading the sections "Kernels for Sequences" (starting at the bottom of page 12) up until "Summary and Further Reading" (in the middle of page 16)
• For all students, it's a good idea (i.e. recommended) to check out the "Summary and Further Reading" section starting at the middle of page 16 and going till the end of the tutorial.
• (Murphy) Chapter 4 (for MLE and MAP estimation)
• This reading is an optional alternative to the Mitchell reading for this topic.
• Read from the beginning up until the end of Section 4.2. However, Section 4.2.6.2 (MLE for the covariance matrix) is more advanced and so I only recommend grad students check it out.
• Read Section 4.5 from the beginning until the end of Section 4.5.1
• (Murphy) Chapters 9 and 10 (for Naive Bayes and Logistic Regression)
• This reading is an optional alternative to the Mitchell reading for this topic.
• Read all of Chapter 9
• Read from the beginning up until the end of Section 10.2.4, but undergrads can skip Section 10.2.3.4 (Deriving the Hessian)
• Read Section 10.2.7 (MAP estimation)
• Read Section 10.3 (Multinomial logistic regression) up till the end of Section 10.3.2.1
• (Murphy, 2012) Chapter 11 (for Gaussian mixture models and EM)
• Note that this is Murphy's 2012 book, "Machine Learning: A Probabilistic Perspective". It is freely available through the UVic library at this link
• Read 11.1 (Latent variable models)
• Read 11.2 (Mixture models), from the beginning up to and including Section 11.2.3
• Read 11.4 (The EM algorithm) from the begining up and and including Section 11.4.2.5