DATASCI 447 Lecture 7: What do Neural Networks Actually Learn?

Administrative Stuff

RECAP: THE CURSE OF DIMENSIONALITY

Stone’s Minimax Theorem: The best achievable rate of convergence is

\[O\left(N^{-\frac{2S}{2S+P}}\right)\]

As \(P\) increases, this rate becomes glacially slow

• \(P = 10\): Need \(\sim 10^{12}\) samples for 1% error
• \(P = 100\): Need \(\sim 10^{102}\) samples for 1% error

No algorithm can escape this — it’s a fundamental property of high-dimensional learning

Question: Is there any hope? What assumption might let us escape this bound?

RECAP: THE MANIFOLD HYPOTHESIS

Real high-dimensional data doesn’t fill the ambient space

The hypothesis: Data lies on \(\mathcal{M}^K\) embedded in \(\mathbb{R}^P\) where \(K \ll P\)

Real images, text, and audio occupy a tiny fraction of the possible space

If we can find \(\phi: \mathbb{R}^P \rightarrow \mathbb{R}^K\) that captures manifold structure, we can learn in \(K\) dimensions instead of \(P\)

Question: How do we actually find this mapping \(\phi\)?

RECAP: THE REPRESENTATION LEARNING FRAMEWORK

┌─────────────┐     ┌────────────┐     ┌─────────────────┐     ┌───────────┐
│ Raw Features│     │ Projection │     │ Representation  │     │  Simple   │
│     ℝᴾ      │ ──▶ │     φ      │ ──▶ │       ℝᴷ        │ ──▶ │ Classifier│
│(e.g., 4096  │     │            │     │  (e.g., 50 dims)│     │(e.g., log)│
│   pixels)   │     │            │     │                 │     │           │
└─────────────┘     └────────────┘     └─────────────────┘     └───────────┘

The heavy lifting is in the projection \(\phi\), not the classifier!

Question: What’s the simplest projection \(\phi\) we could use? Does it have a closed-form solution?

RECAP: PCA AS LINEAR φ

We explored PCA as a linear projection:

Closed-form solution: Top \(K\) eigenvectors of covariance matrix \(S = \frac{1}{N}X^TX\)

Optimal for reconstruction: Minimizes \(\sum_{i=1}^{N} \|x_i - \hat{x}_i\|^2\)

Equivalent to linear autoencoder: Encoder and decoder are tied

\[z = W^T x, \quad \hat{x} = W z\]

Question: PCA is optimal for linear projections and autoencoding. When might it fail? What kinds of transformations can it do?

RECAP: PCA FAILS ON NONLINEAR MANIFOLDS

Swiss roll: PCA rotates but cannot unroll

Rotated horses: Linear interpolation gives ghostly blends

The core problem: Covariance only captures linear relationships

\[\text{Cov}(X_i, X_j) = \mathbb{E}[(X_i - \mu_i)(X_j - \mu_j)]\]

This misses circular patterns, rotations, and nonlinear deformations

THE NEURAL NETWORK ANSWER

Replace linear projection with a learned nonlinear projection:

\[\phi(x) = \varphi(Wx + b)\]

Where:

• \(W \in \mathbb{R}^{M \times P}\) — learned weight matrix
• \(b \in \mathbb{R}^M\) — learned bias vector
  
• \(\varphi(\cdot)\) — elementwise nonlinearity (e.g., ReLU)

The classifier on top is still just logistic regression!

All the representational work happens in learning \(\phi\)

TODAY’S KEY QUESTIONS

1. Why should this work at all? What guarantees that such a \(\phi\) exists?

2. When does this approach actually help? Versus simpler methods like trees?

3. What does the network actually learn? Can we interpret the representation?

THE GOAL: TASK-OPTIMAL REPRESENTATIONS

With PCA, we found \(\phi\) that minimizes reconstruction error

But we don’t care about reconstruction — we care about classification or regression

The new goal: Find \(\phi\) such that a simple classifier achieves low task loss

\[\min_{\phi, \theta} \sum_{i=1}^{N} L(y_i, f_\theta(\phi(x_i)))\]

Where \(L\) is cross-entropy (classification) or MSE (regression)

The representation should be optimized for what we actually want to do

WHAT DOES “OPTIMAL” MEAN FOR CLASSIFICATION?

For classification, the ideal representation makes classes linearly separable

If classes are linearly separable in \(\phi\)-space, logistic regression achieves zero training error

The question: Does such a \(\phi\) always exist?

LINEAR SEPARABILITY IN HIGH DIMENSIONS

Theorem: Any \(N\) points with distinct locations can be perfectly separated by a hyperplane in \(N-1\) dimensions or higher

Intuition: More dimensions = more degrees of freedom for placing a hyperplane

Extreme case: Map each point to its own dimension

\[\phi(x_i) = e_i = (0, \ldots, 0, 1, 0, \ldots, 0)\]

Now any labeling is trivially separable — but this is just memorization!

A SIMPLE EXAMPLE: CONCENTRIC CIRCLES

No line can separate blue from red!

THE KEY INSIGHT: ADD A DIMENSION

What if we add a third dimension based on distance from origin?

\[\phi(x_1, x_2) = (x_1, x_2, x_1^2 + x_2^2)\]

The inner circle has small \(x_1^2 + x_2^2\), the outer circle has large \(x_1^2 + x_2^2\)

CONCENTRIC CIRCLES IN 3D

A horizontal plane now separates the classes!

THE TRANSFORMATION MADE IT POSSIBLE

The transformation \(\phi(x) = (x_1, x_2, x_1^2 + x_2^2)\) “lifted” the data into a space where a hyperplane works

But: We had to know the right transformation! What if we don’t?

THE PROBLEM WITH HAND-CRAFTED FEATURES

For concentric circles, \(\phi(x) = (x_1, x_2, x_1^2 + x_2^2)\) works perfectly

But:

• How did we know to use \(x_1^2 + x_2^2\)?
• What if the pattern is more complex?
• What if we’re in 1000 dimensions?

We need a way to learn the right transformation from data

NEURAL NETWORKS: LEARNING THE TRANSFORMATION

A single hidden layer with \(M\) units:

\[\phi(x) = \sigma(Wx + b), \quad W \in \mathbb{R}^{M \times P}, \quad b \in \mathbb{R}^M\]

This maps \(\mathbb{R}^P \rightarrow \mathbb{R}^M\)

Key insight: The network learns \(W\) and \(b\) to make the classification easy

THE MODEL PATH: FROM LINEAR TO INTERPOLATION

We have a dial that goes from underfitting to interpolation

\(M = 0\): THE LINEAR BASELINE

With no hidden layer:

\[\hat{y} = \text{softmax}(Vx + c)\]

This is just multinomial logistic regression

• Decision boundaries are hyperplanes in input space
• Cannot separate XOR, circles, or any nonlinearly separable data

HOW RELU CREATES “FOLDS”

Each hidden unit computes:

\[h_j = \varphi(w_j^T x + b_j) = \max(0, w_j^T x + b_j)\]

This creates a fold in the input space along the hyperplane \(\{x : w_j^T x + b_j = 0\}\)

• Positive side: unit outputs the linear function
• Negative side: unit outputs zero (folded flat)

THE ADDITIVE STRUCTURE

The full model:

\[f(x) = \sum_{j=1}^{M} v_j \cdot h_j + c = \sum_{j=1}^{M} v_j \cdot \max(0, w_j^T x + b_j) + c\]

\[P(y=1) = \sigma(f(x)) = \frac{1}{1 + e^{-f(x)}}\]

Each hidden unit contributes a scaled fold to the linear predictor

The decision boundary is where \(f(x) = 0\)

M = 1: ORIGINAL RELU OUTPUT

\[h_1 = \max(0, w_1^T x + b_1)\]

One fold — a single tilted plane clipped at zero

M = 1: SCALED RELU OUTPUT

\[v_1 \cdot h_1\]

The coefficient \(v_1\) scales the fold (positive → blue, negative → red)

M = 1: LINEAR PREDICTOR (SUM)

Training accuracy: 45.7%

One fold cannot separate circles!

M = 2: ORIGINAL RELU OUTPUTS

\[h_j = \max(0, w_j^T x + b_j) \quad \text{for } j = 1, 2\]

M = 2: SCALED RELU OUTPUTS

\[v_j \cdot h_j \quad \text{for } j = 1, 2\]

M = 2: LINEAR PREDICTOR (SUM)

Training accuracy: 50.0%

Two folds still not enough!

M = 4: ORIGINAL RELU OUTPUTS

\[h_j = \max(0, w_j^T x + b_j) \quad \text{for } j = 1, 2, 3, 4\]

M = 4: SCALED RELU OUTPUTS

\[v_j \cdot h_j \quad \text{for } j = 1, 2, 3, 4\]

M = 4: LINEAR PREDICTOR (SUM)

Training accuracy: 100.0%

Four folds start to form a pyramid!

M = 9: ORIGINAL RELU OUTPUTS

\[h_j = \max(0, w_j^T x + b_j) \quad \text{for } j = 1, \ldots, 9\]

M = 9: SCALED RELU OUTPUTS

\[v_j \cdot h_j \quad \text{for } j = 1, \ldots, 9\]

M = 9: LINEAR PREDICTOR (SUM)

Training accuracy: 100.0%

Nine folds form a cone approximating the circle!

M = 16: ORIGINAL RELU OUTPUTS

\[h_j = \max(0, w_j^T x + b_j) \quad \text{for } j = 1, \ldots, 16\]

M = 16: SCALED RELU OUTPUTS

\[v_j \cdot h_j \quad \text{for } j = 1, \ldots, 16\]

M = 16: LINEAR PREDICTOR (SUM)

Training accuracy: 100.0%

The \(f=0\) contour is nearly circular!

THE PROGRESSION: MORE FOLDS → BETTER APPROXIMATION

The sum of scaled ReLU folds approximates any shape!

THE REPRESENTATION IS OPTIMIZED FOR THE LOSS

The network doesn’t just find any separating transformation

It finds one that minimizes the classification loss

\[\min_{W, b, V, c} \sum_{i=1}^{N} L(y_i, \sigma(V^T \max(0, W x_i + b) + c))\]

Different losses lead to different learned representations!

A HARDER PROBLEM: WIGGLY DECISION BOUNDARY

The true decision boundary follows \(\sin(x_1^2 - x_2^2) = 0\)

INCREASING WIDTH ON THE WIGGLES PROBLEM

UNIVERSAL APPROXIMATION THEOREM

Theorem (Cybenko 1989, Hornik 1991): For any continuous function \(f\) on a compact domain and any \(\varepsilon > 0\), there exists \(M\), \(W\), \(b\), \(v\), \(c\) such that:

\[\sup_x |f(x) - v^T \varphi(Wx + b) - c| < \varepsilon\]

For classification: With enough hidden units, we can approximate any decision boundary

WHAT THE THEOREM SAYS (AND DOESN’T SAY)

What it guarantees:

• Such a network exists
• Works for ReLU, sigmoid, tanh, and most nonlinearities

What it doesn’t tell us:

• How large \(M\) needs to be (could be enormous)
• How to find the parameters \(W\), \(b\), \(v\), \(c\)
• Whether the solution will generalize to new data

THREE DISTINCT CHALLENGES

A Quick Generalization Preview

A simpler problem:

A Quick Generalization Preview

A Quick Generalization Preview

As the number of hidden units gets too big, we move into overfit territory!

FROM CLASSIFICATION TO REGRESSION

We’ve seen how NNs create decision boundaries for classification

\[f(x) = \sum_{j=1}^{M} v_j \cdot \max(0, w_j^T x + b_j) + c\]

Classification: Decision boundary at \(f(x) = 0\)

Regression: Use \(f(x)\) directly as the prediction!

\[\hat{y} = f(x) = \sum_{j=1}^{M} v_j \cdot \max(0, w_j^T x + b_j) + c\]

THE REGRESSION SETUP

Model: \(\hat{y} = v^T \sigma(Wx + b) + c\)

Loss: Mean squared error

\[L = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2\]

Universal approximation: For any continuous function and any \(\varepsilon > 0\), there exists a network that achieves MSE \(< \varepsilon\) on training data

RELU NETWORKS ARE PIECEWISE LINEAR

Each ReLU unit creates a “hinge” — a point where the slope changes

\[h_j = \max(0, w_j^T x + b_j)\]

For 1D input:

• If \(w_j x + b_j < 0\): output is 0 (flat)
• If \(w_j x + b_j \geq 0\): output is linear with slope \(w_j\)

The hinge is at \(x = -b_j / w_j\)

1D EXAMPLE: A SINGLE RELU UNIT

Each ReLU creates one hinge point — a change in slope

COMBINING RELUS: PIECEWISE LINEAR FUNCTIONS

With \(M\) hidden units:

\[f(x) = \sum_{j=1}^{M} v_j \cdot \max(0, w_j x + b_j) + c\]

• Each unit contributes one potential hinge
• \(M\) units can create up to \(M\) hinges
• The network learns where to place hinges and what slopes to use

BUILDING UP COMPLEXITY: M = 1, 2, 4, 8

More units = more hinges = better approximation

THE TRUE FUNCTION VS. APPROXIMATION

ReLU networks approximate smooth functions with piecewise linear segments

THIS SHOULD FEEL FAMILIAR…

ReLU networks create piecewise linear approximations

What other method creates piecewise approximations?

DECISION TREES: PIECEWISE CONSTANT

Trees create piecewise CONSTANT approximations (step functions)

SIDE-BY-SIDE: NEURAL NET VS. TREE

THE STRUCTURAL COMPARISON

Both are adaptive basis function methods!

2D: HOW THEY PARTITION SPACE

Trees: rectangles | NNs: arbitrary polytopes

WHEN DO TREES WIN?

For tabular, low-dimensional data, tree ensembles often outperform neural networks:

1. Inductive bias: Trees assume axis-aligned splits are meaningful — often true for interpretable features

2. Regularization: Natural controls (max depth, min samples per leaf)

3. Training stability: XGBoost/Random Forests are remarkably robust

4. Parameter efficiency: A depth-10 tree has at most \(2^{10} \approx 1000\) leaves

THE PARAMETER COST OF NEURAL NETWORKS

For a single hidden layer:

• Input: \(P\) dimensions
• Hidden: \(M\) units
  
• Output: 1 (regression) or \(K\) (classification)

Parameters: \(M \times P + M + M + 1 = M(P + 2) + 1\)

For \(P = 100\) features, \(M = 50\) hidden units: 5,101 parameters

Compare to a depth-5 tree: at most 31 splits to learn

AN HONEST COMPARISON: TABULAR DATA

On tabular data, gradient boosting is hard to beat!

WHY TREES WIN ON TABULAR DATA

Axis-aligned splits make sense:

• “Income > $50K” is interpretable
• “0.73 × Age + 0.21 × Income - 0.44 × Debt > 1.2” is not

Features are already meaningful:

• Age, income, location — these have semantic meaning
• No need to learn a transformation

Trees exploit this structure; NNs must learn it

SO WHEN DO NEURAL NETWORKS WIN?

When the inductive bias of trees is wrong:

• Data has nonlinear interactions that aren’t axis-aligned
• Features are raw measurements without semantic meaning
• Data lies on a low-dimensional manifold in high dimensions

Key examples: Images, audio, text

THE MANIFOLD ADVANTAGE

Stone’s theorem: Convergence rate is \(O(N^{-2S/(2S+P)})\)

For \(P = 4096\) (64×64 image): astronomical sample complexity

But if data lies on a \(K\)-dimensional manifold with \(K \ll P\):

• Trees partition the ambient space \(\mathbb{R}^P\) — exponentially many cells needed
• NNs can learn to project onto the manifold — only \(K\) effective dimensions

This is exactly what happens with images, audio, and text

THE MANIFOLD HYPOTHESIS (REVISITED)

High-dimensional data often lies on a low-dimensional manifold

Images: 64×64 grayscale = 4,096 dimensions

But the space of “valid faces” is much smaller:

• Lighting (a few dimensions)
• Pose (a few dimensions)
• Expression (a few dimensions)
• Identity (more dimensions, but still far fewer than 4,096)

WHY TREES FAIL ON IMAGES

Trees ask questions like:

“Is pixel (32, 47) > 128?”

But semantic content isn’t axis-aligned:

• A smile involves relationships between pixels
• The same expression can appear at different locations
• Rotation, scale, lighting change all pixel values

Trees partition the ambient space — exponentially many splits needed

WHY NNS CAN SUCCEED

Neural networks learn a transformation \(\phi(x)\) that:

1. Maps raw pixels to a meaningful representation
2. Is optimized for the task (classification, regression)
3. Can capture nonlinear relationships between pixels

If the transformation finds the manifold, we escape the curse of dimensionality

DEMONSTRATION: THE OLIVETTI FACES DATASET

Classic dataset for face recognition:

• 40 people, 10 images each (400 total)
• 64 × 64 grayscale (4,096 pixels)
• Variations: lighting, expression, glasses

Dataset shape: (400, 4096)
Number of classes: 40
Images per person: 10

SAMPLE FACES FROM OLIVETTI

THE CHALLENGE

• Input dimension: \(P = 4,096\)
• Training samples: ~320 (80% of 400)
• Classes: 40

Stone’s bound suggests this should be very hard

But the “face manifold” is much lower dimensional…

RECALL: EIGENFACES (PCA)

Variance explained by top 50 PCs: 87.4%

PCA captures global structure but treats all variation equally

OLIVETTI RESULTS

WHAT THE NEURAL NETWORK LEARNS

The hidden layer learns a task-specific representation

Unlike PCA (which optimizes for reconstruction), the NN optimizes for classification

VISUALIZING FIRST-LAYER WEIGHTS

The network learns face-like patterns (edges, regions, structures)

SCALING UP: FER2013

Olivetti is small (400 images). What about a larger, harder dataset?

FER2013: Facial Expression Recognition

• ~35,000 images (48 × 48 grayscale)
• 7 classes: Angry, Disgust, Fear, Happy, Sad, Surprise, Neutral
• Real-world variability in pose, lighting, identity

FER2013: THE CHALLENGE

This is a manifold problem:

“Happy” isn’t a fixed pixel pattern — it’s a configuration:

• Corners of mouth turned up
• Cheeks raised
• Eyes crinkled

This configuration can appear on any face, at any position, under any lighting

WHY TREES STRUGGLE ON FER2013

Decision trees ask: “Is pixel (23, 31) > 128?”

But expression lives in relationships:

• Mouth corners vs. mouth center
• Eye shape relative to eyebrows
• Cheek position

These relationships are not axis-aligned

Trees would need exponentially many splits to capture them

THE INDUCTIVE BIAS STORY

Fully-connected NNs are more general than trees but less structured than CNNs

WHAT WE’VE ESTABLISHED

1. The Goal: Learn a representation \(\phi(x)\) that makes classification/regression easy

2. Universal Approximation: A single hidden layer with enough units can approximate any function

3. The Mechanism: ReLU units create “folds” that combine additively

4. Width as Complexity: \(M=0\) (linear) → \(M=\infty\) (interpolation)

5. When NNs Will Work Best: High-dimensional data with manifold structure (images, audio, text) with an appropriate prior architecture

BUT WAIT…

On FER2013, we saw something interesting:

Deeper networks did slightly better — even with similar parameter counts!

Why?

THE QUESTION OF DEPTH

Universal approximation says: width is enough

With sufficient hidden units, one layer can approximate anything

But it says nothing about efficiency:

• How many units do we need?
• Is there a better way?

A THOUGHT EXPERIMENT

Consider learning to recognize “a person wearing glasses and smiling”

Single layer approach:

• Must learn the complete pattern directly from pixels
• Every combination of (glasses shape × face shape × smile type × lighting × pose)
• Exponentially many patterns to memorize

A THOUGHT EXPERIMENT

Multi-layer approach:

• Layer 1: Detect edges
• Layer 2: Detect face parts (eyes, mouth, glasses frames)
• Layer 3: Detect configurations (glasses present, smile present)
• Layer 4: Combine into final decision

COMPOSITIONAL STRUCTURE

Real-world data has hierarchical structure:

Images: \[\text{Pixels} \rightarrow \text{Edges} \rightarrow \text{Textures} \rightarrow \text{Parts} \rightarrow \text{Objects}\]

Language: \[\text{Characters} \rightarrow \text{Words} \rightarrow \text{Phrases} \rightarrow \text{Sentences} \rightarrow \text{Meaning}\]

Audio: \[\text{Samples} \rightarrow \text{Frequencies} \rightarrow \text{Phonemes} \rightarrow \text{Words} \rightarrow \text{Speech}\]

Can depth help us exploit this structure?

PREVIEW: THE DEPTH EFFICIENCY ARGUMENT

Theorem (informal): Some functions require exponentially many units with 1 hidden layer, but only polynomially many with multiple layers.

Example: Parity function on \(n\) bits

• 1 hidden layer: requires \(2^{n-1}\) units
• \(n\) hidden layers: requires \(O(n)\) units

Depth can be exponentially more efficient than width!

WHAT INTERMEDIATE LAYERS LEARN

In deep networks trained on images:

• Layer 1: Edge detectors
• Layer 2: Combinations of edges (corners, textures)
• Layer 3: Object parts (eyes, wheels, handles)
• Layer 4+: Whole objects and scenes

Each layer builds on the previous — compositional representation

THE COST OF DEPTH

If depth is so great, why not make everything deep?

Optimization challenges:

1. Vanishing gradients: Signals shrink as they propagate backward
2. Exploding gradients: Signals grow uncontrollably
3. Harder to train: More local minima, longer training

These problems were the bottleneck for neural networks until ~2012

PRACTICAL TAKEAWAYS

1. Start simple: Try logistic regression and XGBoost first

2. Use NNs when: High-dimensional, raw features, manifold structure

3. Width controls capacity: More units = more flexibility = more risk of overfitting

4. Depth often helps: But makes training harder (we’ll learn why)

5. The representation is key: What the hidden layer learns matters more than the architecture details

NEXT TIME

Width vs. Depth: The Efficiency of Deep Networks

• Theoretical results on depth separation
• What intermediate layers learn
• The compositionality hypothesis
• Visualizing deep representations