Course Schedule
(Week 1, Week 2, Week 3, Week 4, Week 5, Week 6, Week 7, Week 8, Week 9, Week 10, Week 11)
Week 1
Thursday (September 22): Course Introduction
- Topics (Session Slides):
- Getting acquainted
- Course overview, goals, and administrivia
- Introduction to Data Mining
- Overview/goals of data mining (DM) and knowledge discovery (KD)
- Myths about data mining
- Readings:
- TB: Chapters 1 & 2
- SAS Data Mining Certificate at Fisher
- Data, data everywhere, The Data deluge, The Economist, 2/10.
- A Golden Vein, The Economist, 1/04.
- Cases:
- Diamonds in the Data Mine, HBR, 5/03
- How Verizon cut Customer Churn, Financial Express, 10/03.
- Hard Hats for Data Miners: Myths and Pitfalls of Data Mining, DMReview, 5/05.
- Skim:
- Resources:
- Online Data Mining Portals: KDnuggets, The Data Mine
Week 2
Tuesday (September 27): The Data Mining Process - Data Extraction and Manipulation
- Topics (Session Slides):
- Overview of the Data Mining Process
- The Relational Data Model and Relational DBMS
- Enterprise Reporting
- Relational Algebra
- SQL: The Relational Query Language
- Readings:
- TB: Chapter 2
- CRISP-DM Process Model
- Resources:
- MS Access (2003) Primer
- MS Access 2007 Quick (Interface) Tutorial: 1, 2.
- Interactive SQL Tutorial
Thursday (September 29): Data Extraction and Manipulation
- Topics:
- Database Schemas and Instances
- Principles of Query Formulation
- Database Definition and Manipulation in MS ACCESS
- Query formulation (Demo DB)
- Readings:
- Skim through chapters 1-4 in the online access tutorial book.
- Assignments:
- Download and Complete the MS Access Lab Tutorial.
- Assignment 1 -- Data Manipulation Using Access (due Tuesday, October 4)
Week 3
Tuesday (October 4): Data Warehousing
- Topics (Session Slides):
- The Case for Datawarehousing
- Building a datawarehouse
- Readings:
- H. Watson and B. Haley, "Datawarehousing: Managerial Considerations," Communications of the ACM, Vol. 41, No. 9 (Sept. 1998), Pages 32-37.
- Related Links/Resources:
Thursday (October 6): OLAP and Data Exploration
- Topics (Session Slides):
- Review solution to Assignment 1.
- Multidimensional Databases
- On-Line Analytical Processing
- Demo - Pivot Tables
- Data Exploration
- Review of Descriptive Statistics and Probability Concepts (Slides)
- Readings:
- An Introduction to OLAP Multidimensional Terminology and Technology
- Video: Using Pivot Tables (in Access 2007)
- Basic Probability Notes
- Related Links/Resources:
Week 4
Tuesday (October 11): Association Rule Mining
- Topics (Session Slides):
- Market Basket Analysis and Other Applications
- Frequent Itemset and Association Rule Mining
- Rule Support & Confidence
- Readings:
- TB: Chapter 13
- Related Links (FYI):
Thursday (October 13): Association Rule Mining (Continued)
- Topics:
- Apriori Algorithm
- Rule Evaluation
- Sequential patterns
- Mining for Association Rules using XLminer
- Readings:
- R. Agrawal and R. Srikant, "Fast Algorithms for Mining Association Rules," Proc. 20th Int. Conf. Very Large Data Bases (VLDB), 1994. (FYI; only skim)
- Assignments:
- Assignment 2 -- Association Mining (due Tuesday, October 18)
Week 5
Tuesday (October 18): Cluster Analysis
- Topics (Session Slides):
- Segmentation and Personalization
- Similarity Measures
- The K-means algorithm (Excel Spreadsheet Demo)
- Hierarchical Clustering
- Cluster Validation and Interpretation
- Readings:
- TB: Chapter 14
- Related Links (FYI):
Thursday (October 20): Cluster Analysis (Continued)
- Topics:
- Cluster Evaluation
- Clustering using XLminer
- Demo: Synthetic Dataset
- Assignments:
- Assignment 3 -- Cluster Analysis (due Thursday, October 27)
Week 6
Teusday (October 25): Cluster Analysis (Ctd.)
Thursday (October 27): Principal Component Analysis
- Topics (Session Slides):
- Data Reduction using Principal Component Analysis
- Review of Assignment 3 on Cluster Analysis. Solution: 14.2_Pharmaceuticals (Spreadsheet), 14.3_Cereals (Spreadsheet)
- Read:
- Chapter 4
Week 7
Tuesday (November 1): Midterm Exam
Thursday (November 3): Predictive Modeling - Classification
- Topics (Session Slides):
- General Approach to Solving Classification Problems
- Decision Tree Induction
- Readings:
- Textbook: Chapter 9
- Assignments:
- Assignment 4 (due Tuesday, November 8)
- Related Links (FYI):
Week 8
Tuesday (November 8): Model Evaluation
- Topics (Session Slides):
- More on Classification
- Overfitting and Underfitting
- Accuracy & Recall
- Classification (Confusion) Matrix
- Readings:
- Textbook: Chapter 5
Thursday (November 10): More on Model Evaluation & Response Modeling
- Topics:
- Building Decision Tree Models in XLMiner
- Response Modeling (Handout)
Week 9
Tuesday (November 15): Response Modeling (Ctd.) and Regression Trees
- Topics:
- Lift Charts
- Building Classification/Decision Tree Models
- Building and Evaluating Regression Tree Models
- Assignments:
- Assignment 5 -- Predictive Modeling (due November 18)
Thursday (November 17): Predictive Modeling Using Regression
- Topics (Session Slides):
- Introduction to OLS Regressions
- Simple and Multiple Linear Regression
- Variable Selection and Stepwise Regression
- Model interpretation
- Readings:
- Chapter 6
Week 10
Tuesday (November 22): Predictive Modeling Using Logistic Regression
- Topics (Session Slides):
- Multiple Regression (Ctd.)
- Logistic Regression
- Model Evaluation
- Building LR Models in XLMiner
- Solution to Assignment 5: 7.1_eBayAuctions-DT (Spreedsheet)
- Demo: Solution using Logistic Regression: 8.4_eBayAuctions-LR (Spreedsheet)
- Readings:
- Chapter 10
Thursday (November 24): No Class - Happy Thanksgiving!
Week 11
Tuesday (November 29): Predictive Modeling Using Neural Networks & Ensamble Methods
- Topics (Session Slides: Neural Networks, Ensambles)
- Introduction to Neural Networks
- Neural Networks vs. Regression
- Ensamble Methods
- Readings:
- Chapter 11
- Related Links (FYI):
- Introduction to Neural Networks
- Classification-based: Forecasting Fraudulent Financial Statements using Data Mining (IJCI 2006).
Thursday (December 1): Project Presentations
Tuesday (December 6): Final Exam