DATASCI 447 Lecture 1: Introduction
Kevin McAlister
January 13, 2026
Welcome to DATASCI 447!
This is a class that is going to provide an overview of the statistical methods that underpin a number of more advanced machine learning techniques
- The goal of this class is to provide a statistical/mathematical introduction to topics in machine learning that extend on those methods learned in your introductory class
Class flow:
Part 1: Machine Learning as Distributional Matching, The Manifold Theorem
Part 2: Neural Networks and Deep Learning as Representation Algorithms
Part 3: Generative Machine Learning (or making cool pictures of dogs and putting hats on celebrities)
Pre-requisites
For this class, I am expecting that students have taken a class on:
Linear regression (Like DATASCI 220)
Machine Learning (like DATASCI 347)
Calculus 1-3
Linear algebra
I don’t expect you to remember everything from these classes, but I do expect that when you see certain concepts they aren’t completely foreign.
Lectures
We’re going to meet here on Tuesdays and Thursdays from 2:30 PM - 3:45 PM
There is no formal attendance requirement for this course, but you should show up
Be willing to stop me if there’s something that isn’t clear. If it’s not clear to you, then I’m sure there’s someone else in class who is also confused.
It’s a relatively small class, so I have no problem spending time re-clarifying points that may not have been well made.
Office Hours
I’ll have two office hours periods this semester:
Wednesdays 2:30 - 3:45
And one other time that I’ll announce next week
All office hours will occur in my office - PAIS 579
Group Problem Sets
Over the course of the semester, there will be 6-7 problems sets. These will account for 50% of your final grade.
Implement and extend the materials discussed in class.
Introduce software and coding
Derivations and Proofs
Problem Sets will be where you get your applied practice
Lectures will largely center on the theory of why things work
Problem sets will relate more to the how
Group Problem Sets
All problem sets should be submitted to the appropriate Canvas assignment as two files:
The raw notebook file (.qmd, .ipynb)
A rendered version of the notebook (.html or .pdf)
Each problem set will be posted in a variety of different formats, so feel free to use this as a template for your final solutions.
Group Problem Sets
All problem sets in this class can be completed in groups of, at most, 3 students
Each student should turn in a copy of the solutions, but can be identical to group’s solutions.
All collaborators must be outlined at the top in the by-line
I recommend the same group each time - repeated vs. one-shot games of trust.
Since it is an upper level course, I’m creating a mechanism for getting rid of shirkers
- If they don’t contribute equally, get rid of ’em.
You can complete assignments individually.
I can also assign you a group, if you’d like. Just email me and I’ll put people together who want a group.
Final Project
The other 50% of your final grade will be determined by a final project
A significant project that applies methods discussed in this class
Up to you (and your group) exactly what this is
Do something interesting!
Projects that are too simple will not be accepted
No basic comparisons of methods
Really try to answer a question that’s interesting to you
Final Project
Three checkpoints:
On March 26th, teams will present a single slide outlining their final project (10%)
On April 25th, teams will present project posters at QTM’s end of semester showcase (20%)
By May 5th, each student should submit a final paper about the final project. A scientific-styled paper no more than 15 pages and a nicely formatted Github codebase (10% for deliverable, 10% for Github)
Final Project
The goal of this final project is to give you something to include in your portfolio as you apply for jobs or grad school
- Maybe start something that turns into a bigger project later
Coding
I’ll be using Python for this class.
You can technically use any language you want for this class
But, I highly recommend Python
Deep learning libraries are developed mostly with Python in mind
Coding
My setup:
Python 3.10
VSCode
Jupyter Notebooks/Quarto
My local hardware (my beefy little baby):
AMD Ryzen 5900x - 12 core/24 thread processor
NVIDIA RTX 5090 - 21,760 CUDA Cores/32GB GDDR7 VRAM
96GB DRAM
Coding
As we progress through this class, we’re going to get to methods that are computationally demanding
- Often too demanding for your laptop
If you don’t have a machine with a discrete GPU, you’ll be using Google Colab
I highly recommend paying $10 a month for Colab Pro during this class
Gives access to better GPUs and priority time on them
If $10 a month presents a problem, let me know.
Coding
We’ll be using PyTorch throughout this class
Software for general purpose optimization
The dominant software for deep learning
Using nice utility functions from PyTorch Lightning
A really nice library that sits over PyTorch that organizes code
Automatically detects and uses GPUs and other accelerators when available
Coding
Please use Github Copilot/Codex/Etc.!
- This is a machine learning course an these LLMs are a modern marvel!
So many of the annoyances of coding with Python are gone when you use code LLMs
Matplotlib syntax
Documentation searches
scikit-learn nuances
Pytorch modules and training loops
If you have a Github account, get it student certified and you’ll be able to use Copilot for free
- I think the $20 ChatGPT pro license/Google AI license is money well spent
The Goals of this Class
The topics I hope to cover this semester are outlined on the syllabus.
The pace is ambitious.
If we need to take more time on topics, we will adjust.
The Goals of this Class
Quick note: I’m trying something new this semester and changing the lens through which I’m presenting the materials in this class to make generative machine learning a common thread
- Not your typical deep learning class
A part of this is that optimization doesn’t fit in to our lecture flow all that well
I’ve written a webpage that talks a little about matrix calculus, gradents/Hessians, optimization, gradient descent, and stochastic gradient descent
I’m hoping that it will be sufficient for you to go through this material on your own
By the beginning of week 4, I’m expecting that you’ve at least looked over this material and have a broad understanding.
The Goals of this Class
Any questions?
Machine Learning
What is a machine learning algorithm?
A common definition (Mitchell, 1997):
“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”
In other words, a machine learning algorithm learns from data - the more data and the more trials, the better
We would hope that the algorithm is able to perform well at a defined task given some data
Tasks
Machine learning is so popular because it can be used to address many different questions/achieve many different tasks.
Two basic supervised starting points:
Regression - given a vector of input features, \(\mathbf X\), and an outcome vector, \(\mathbf y \in \mathbb R\), output a function \(\hat{f}(\mathbf x)\) that accurately predicts the value of the outcome given \(\mathbf x\).
- Use inputs to predict the value of a continuous outcome.
What are some questions we can answer with regression algorithms?
Tasks
Machine learning is so popular because it can be used to address many different questions/achieve many different tasks.
Two basic supervised starting points:
Classification - given a vector of input features, \(\mathbf X\), and an outcome vector, \(\mathbf y \in (1,2,...,K)\), output a function \(\hat{f}(\mathbf x)\) that accurately predicts the class given \(\mathbf x\).
- Use inputs to predict the value of a countable and finite set of outcomes.
What are some questions we can answer with classification algorithms?
Tasks
These two basic tasks largely describe what you’ve seen previously.
In your introductory machine learning class, the methods discussed largely centered on solving these kinds of problems.
We have a feature matrix ( \(N\) observations of \(P\) features), \(\mathbf X\)
We have an outcome vector ( \(N\) observations of the outcome), \(\mathbf y\)
Use \(\mathbf X\) and \(\mathbf y\) to learn \(\hat{f}(\mathbf x)\) that maps features to a predicted outcome.
\(f(\mathbf x) \approx y\)
Tasks
How do we get from there to here?
Tasks
Or here?
Tasks
Or here?
Tasks
Or here?
Note: This is an at home listen. Also, apologies for the language. But, this is the best AI generated music in the game right now!
The World is A Simulation?
Unfortunately, this toolkit will only get us so far
- We need something else
Our old definition of statistical learning:
Learning is finding a function \(f(\mathbf x) \approx y\)
A new view:
Learning is reverse-engineering the mechanism that created the data
The World is A Simulation?
Imagine a Nature Box that outputs data points
Collections of pixels in an image and the label
Words in the written English language
This box follows some distribution
\[ \{\mathbf x, y\} \sim P(X,Y) \]
which is a proper probability distribution over all things
Distribution of pixels in images of animals and the animal label
The sequence of words in a novel (1st word, 2nd word, etc.)
The World is A Simulation?
We don’t know \(P(X,Y)\)
But we can take samples
Pull the lever and one instance drops out!
The distributional goal:
Build a mathematical clone of the box from samples
- If our model in indistinguishable from nature, we have solved the problem!
The World is A Simulation?
The joint distribution \(P(X,Y)\) is the holy grail
If I knew this, I could answer any question about the data (at least up to probability)
How does pixel 1 correlate with pixel 277?
If I say “The dog is”, what is the probability that “barking” is the next word?
What is the distribution of pixels in an image that contains a cat?
The problem is that this distribution is often high dimensional and scary looking
The Fork in the Road
\[ P(X,Y) = P(Y | X) \cdot P(X) \]
Path A (The Discriminative Path): We only care about \(P(Y | X)\) and we throw away \(P(X)\)
Shortcuts the problem. Perfectly fine if our end goal is predicting \(y\) given some set of input features.
Useless for simulation
Path B (The Generative Path): We model \(P(X)\) directly
Harder
Requires modeling the geometry of the data manifold
Black and White Images
Grayscale images are just a matrix of pixel values
Each pixel is between 0 (black) and 255 (white)
Often renormalized between 0 and 1
RGB Images
RGB Images are three matrixes stacked on each other
- Each pixel is a combination of three pixel values between 0 and 255
RGB Images
CIFAR-10 Images: 32 x 32 RGB Images