Non-parametric density estimation

Parzen windows & k-NN: two ways to pick a window

Both methods estimate a class from the same idea: p̂(x) ≈ k / (n·V). A Parzen window fixes the window's volume V and counts how many points k land inside. k-NN fixes k and grows the window until it holds exactly that many.

Training data and decision surface

Click a point to inspect its window (leave-one-out); click empty space to test a new point. The bar strips are 1D histograms of each feature, using the same h as the bin width.

2 features · 2 classes

Weekly practice time and quiz score.

Choosing the window

Leave-one-out accuracy across the window range

Too small a window overfits to single points (high variance); too large a window blurs classes together (high bias).

Two views of the same idea. Both methods classify by the majority label among the points inside a local window; the window is drawn as an ellipse because the two features are normalized to a comparable 0–1 scale before distance is computed. Real implementations often use smoother kernels (Gaussian, Epanechnikov) instead of a hard-edged window, and weight neighbors by distance.