Facebook LinkedIn Discord Youtube E-mail

Notes

Algorithms ▸
K-Nearest Neighbors Algorithm - bjelDark

Animation | Rigging | UE | Unity | Scripts | Photography | MISC

Category selection

K-Nearest Neighbors

Simple classification algorithm.

Classifies target based on closest datapoints.

Calculates distances between target and each datapoint, and then picks top K number of closest datapoints.

From those picked datapoints, chooses class with most entries, and assigns it to the target.

If it's tie between multiple classes, there are various ways to fix it:

1. Try to minimize ties by using an odd k (3, 5, etc.).

2. If a tie still occurs, use distance-weighted voting for deterministic, data-driven tie-breaking:

  - weight class votes by their inverse distance to the target

   weight = 1 / distance + ϵ     [ϵ is small positive number, to avoid division by 0]

3. If domain has class priorities, use that to break ties (pick "fraudulent", "more serious diagnostic", etc.).

4. Randomly pick between tied classes (keeps system unbiased).

  K-Nearest Neighbors [Python Implementation]: