Welcome to MATH 4/5388 – Machine Learning Methods

• Course Numbers: MATH 4388 and MATH 5388

• Course Name: Machine Learning Methods

• Department of Mathematical and Statistical Sciences

• University of Colorado Denver

• Instructor: Farhad Pourkamali, Ph.D.

This course focuses on learning from data and making predictions with minimal human intervention. You will explore fundamental concepts and techniques in machine learning and apply them to real-world problems through hands-on exercises. Using real-world datasets, you will build and evaluate models with tools such as Python, scikit-learn, NumPy, and Pandas.

Modern machine learning workflows integrate mathematics and code, and libraries like NumPy make it intuitive to translate mathematical operations (e.g., matrix and vector products) into clear, efficient code. By engaging with both theory and practical examples, you will strengthen your mathematical understanding while gaining the skills to implement models ranging from simple linear approaches to more advanced methods.

Throughout this course, you will learn about parametric regression and classification models, including linear and logistic regression, along with appropriate evaluation metrics. We will explore the concepts of bias and variance in machine learning methods and discuss the use of regularization techniques. The course will also cover advanced classical machine learning methods such as support vector machines, decision trees, and ensemble techniques like random forests. We will also introduce the fundamentals of unsupervised learning, focusing primarily on clustering methods. The course will conclude with an introduction to neural networks, including their underlying mathematics and basic implementation.

These lecture notes are accompanied by short educational videos. Watch the full playlist here: YouTube Playlist.

Acknowledgment: This project is sponsored by the Colorado Open Educational Resources Program through the Colorado Commission on Higher Education (CCHE).

• The Machine Learning Landscape
• 1. Definition of Machine Learning
• 2. Components and Types of Machine Learning
• 3. Mathematical Concepts Behind Machine Learning
• 4. Programming Concepts for Machine Learning
• Data Preparation and Validation Techniques
• 1. Data Loading, Understanding, and Filtering
• 2. Feature Engineering
• 3. Data Splitting and Cross-Validation
• Cross-Validation
• 4. Pipelines for Automated Preprocessing
• Parametric Regression Models
• 1. Linear regression: Problem formulation, assumption, loss function, gradient
• 2. Linear regression in Scikit-learn
• 3. Evaluation metrics
• 4. Gradient descent (GD) and variants
• 5. Nonlinear extension and regularization
• Parametric Classification Models
• 1. Foundations of Logistic Regression
• 2. Logistic regression in Scikit-learn (sklearn)
• 3. Evaluation metrics
• 4. Multiclass classification
• Unsupervised Learning
• 1. K-means clustering
• 2. DBSCAN
• 3. Gaussian Mixtures
• 4. Evaluating Clustering with Normalized Mutual Information (NMI)
• Support Vector Machines
• 1. SVM Problem Formulation
• 2. Creating an SVM model using Scikit-Learn
• 3. SVMs for regression
• Decision Trees and Random Forests
• 1. Structure of Decision Trees
• 2. Gini Impurity
• 3. CART in Scikit-learn
• 4. Bagging (Bootstrap Aggregating)
• Gradient Boosting
• 1. Bagging (Bootstrap Aggregating) Algorithm
• 2. Mean Decrease Impurity (MDI)
• 3. Boosting
• Mathematical Building Blocks of Neural Networks
• 1. Feedforward networks
• 2. A tour of activation functions
• Building Neural Networks in Keras
• 1. Backprop
• 2. Implementing MLPs with Keras