k-Nearest Neighbors: Loan Approval

1 · Vote by hand

Measure the distance to every training customer, sort, take the k closest, let them vote. This is what predict() hides. Drag the blue × (or use the arrow keys) to move the new customer.

The three presets after [4, 5] are the "extension" customers from the lecture. Try [4, 5] with k = 1 to 6: the prediction flips. Ties in distance are broken by the earlier row (a stable sort, like np.argsort here).

2 · Decision regions: paint your own data

Each small cell of the plane is coloured with the class a kNN model would predict there: that is meshgrid + predict + contourf. Click on empty space to add a customer, click an existing one to remove it.

Try this: load the 80 random applications and compare k = 1 with k = 15. Small k carves little islands around single points (training accuracy 1.00, but leave-one-out accuracy lower); large k gives a smooth boundary. The 80 applications are generated here, not the practice notebook's data. With an even k, ties in the vote go to class 0, so use odd k for two classes.

3 · The training accuracy trap

"Is k = 1 the better model because its accuracy is 1.00?" Compare accuracy on the training data (each customer is its own nearest neighbor) with leave-one-out accuracy: predict each customer from the other five, as if it were new.

Leave-one-out needs k ≤ 5 here (only five customers are left). With six customers this is a toy: the point is the method (judge a model on points it has not seen), not the ranking of k. A real test set, as in the practice, does the same job.

4 · Ties

Two kinds of ties, with different rules.

With two classes, an even k can end in a tie. np.bincount(...).argmax() returns the first index with the maximum count, i.e. the smaller label, 0 (denied). That is why k = 6 gives "denied", as in the lecture, and why odd k is the usual choice.

The two nearest are [4, 4] itself (approved, distance 0) and [3, 4] (denied, distance 1). For the third place, four customers are tied at distance √5 ≈ 2.24. Choose which one is taken:

Which tied customer comes third is an implementation detail. In a check with the current scikit-learn, kneighbors returned row 1 ([2, 5], denied), so [4, 4] was classified as denied and k = 3 scored 0.83. Had a tied approved customer been taken, the accuracy would be 1.00. The other five customers are classified correctly either way.