I've been going from post to post trying to solve this but can't find anything that matches my exact description.
I'm attempting to find the K=1-3 Nearest Neighbors to every X/Y coordinate in a dataset with criteria in which neighbors to compare to.
In Excel I was able to do this using the following formula:
{=INDEX($B$2:$B$216,
MATCH(SMALL(SQRT((IF($A$2:$A$216=$A2,$C$2:$C$216)-$C2)^2+(IF($A$2:$A$216=$A2,$D$2:$D$216)-$D2)^2),E$1+1),
SQRT((IF($A$2:$A$216=$A2,$C$2:$C$216)-$C2)^2+(IF($A$2:$A$216=$A2,$D$2:$D$216)-$D2)^2),0),1)}
This works as follows:
INDEX($B$2:$B$216 returns the final value as ADJ_I
SMALL(SQRT((IF($A$2:$A$216=$A2,$C$2:$C$216)-$C2)^2+(IF($A$2:$A$216=$A2,$D$2:$D$216)-$D2)^2),E$1+1),
SQRT((IF($A$2:$A$216=$A2,$C$2:$C$216)-$C2)^2+(IF($A$2:$A$216=$A2,$D$2:$D$216)-$D2)^2)
This is the Euclidean distance formula with the added Criteria of only comparing values that are within the same LOC_C.
E$1+1 at the end of the first part makes sure that it does not return its own row value.
I am quite new overall to statistical analysis in Python whether its using Pandas or a KDBTree I've tried piecing code together but simply don't have the expertise to make it work for my data. The data and calculations are over 100k rows, so Excel either crashes or simply takes nearly a day to run the full dataset as seen in part below.
Here is a snippet that can be pasted into R:
structure(list(LOC = c(3, 3, 3, 3, 3, 3, 3), ADJ_I = c(1.1, 2.1,
3.1, 3.2, 4.1, 5.1, 6.1), X = c(467, 231, 349, 436, 466, 498,
582), Y = c(381, 252, 208, 226, 312, 292, 249), K_1 = c(29.1,
34.1, 32.1, 7.1, 16.1, 38.1, 37.2), K_2 = c(18.1, 10.1, 17.1,
16.1, 29.1, 16.1, 35.1), K_3 = c(18.1, 10.1, 17.1, 16.1, 29.1,
16.1, 35.1)), class = "data.frame", row.names = c(NA, -7L))
More of the dataset looks like this:
+-------+-------+--------+--------+------+------+------+
| LOC_C | ADJ_I | X | Y | K=1 | K=2 | K=3 |
+-------+-------+--------+--------+------+------+------+
| 3 | 1.1 | 467 | 381 | 29.1 | 18.1 | 18.1 |
| 3 | 2.1 | 231 | 252 | 34.1 | 10.1 | 10.1 |
| 3 | 3.1 | 349 | 208 | 32.1 | 17.1 | 17.1 |
| 3 | 3.2 | 436 | 226 | 7.1 | 16.1 | 16.1 |
| 3 | 4.1 | 466 | 312 | 16.1 | 29.1 | 29.1 |
| 3 | 5.1 | 498 | 292 | 38.1 | 16.1 | 16.1 |
| 3 | 6.1 | 582 | 249 | 37.2 | 35.1 | 35.1 |
| 3 | 7.1 | 470 | 246 | 16.1 | 3.2 | 3.2 |
| 3 | 8.1 | 208 | 260 | 25.1 | 25.3 | 25.3 |
| 3 | 8.2 | 244 | 239 | 25.2 | 10.1 | 10.1 |
| 3 | 8.3 | 235 | 270 | 12.1 | 25.3 | 25.3 |
| 3 | 9.1 | 341 | 298 | 11.4 | 11.3 | 11.3 |
| 3 | 9.2 | 329 | 353 | 42.3 | 42.2 | 42.2 |
| 3 | 10.1 | 226 | 241 | 25.2 | 2.1 | 2.1 |
| 3 | 11.1 | 289 | 265 | 11.2 | 23.1 | 23.1 |
| 3 | 11.2 | 302 | 271 | 11.1 | 23.1 | 23.1 |
| 3 | 11.3 | 331 | 290 | 9.1 | 11.4 | 11.4 |
| 3 | 11.4 | 343 | 307 | 9.1 | 11.3 | 11.3 |
| 3 | 11.5 | 400 | 307 | 42.4 | 14.2 | 14.2 |
| 3 | 11.6 | 383 | 353 | 43.1 | 11.5 | 11.5 |
| 3 | 12.1 | 243 | 276 | 8.3 | 25.3 | 25.3 |
| 3 | 12.2 | 227 | 323 | 15.1 | 12.3 | 12.3 |
| 3 | 12.3 | 218 | 361 | 39.1 | 12.2 | 12.2 |
| 3 | 13.1 | 392 | 436 | 14.3 | 43.1 | 43.1 |
| 3 | 14.1 | 257 | 370 | 42.2 | 39.1 | 39.1 |
| 3 | 14.2 | 423 | 290 | 42.4 | 11.5 | 11.5 |
| 3 | 14.3 | 442 | 433 | 24.1 | 27.1 | 27.1 |
| 3 | 15.1 | 218 | 301 | 12.2 | 25.3 | 25.3 |
| 3 | 16.1 | 464 | 277 | 7.1 | 4.1 | 4.1 |
| 3 | 17.1 | 374 | 227 | 3.1 | 20.1 | 20.1 |
| 3 | 18.1 | 515 | 389 | 31.1 | 21.1 | 21.1 |
| 3 | 19.1 | 288 | 226 | 11.1 | 32.1 | 32.1 |
| 3 | 20.1 | 352 | 260 | 11.3 | 41.1 | 41.1 |
| 3 | 21.1 | 533 | 355 | 31.1 | 18.1 | 18.1 |
| 3 | 22.1 | 549 | 302 | 38.1 | 36.1 | 36.1 |
| 3 | 23.1 | 281 | 284 | 11.1 | 11.2 | 11.2 |
| 3 | 24.1 | 463 | 431 | 27.1 | 14.3 | 14.3 |
| 3 | 25.1 | 218 | 250 | 10.1 | 2.1 | 2.1 |
| 3 | 25.2 | 235 | 240 | 8.2 | 10.1 | 10.1 |
| 3 | 25.3 | 224 | 273 | 8.3 | 12.1 | 12.1 |
| 3 | 26.1 | 510 | 435 | 27.1 | 18.1 | 18.1 |
| 3 | 26.2 | 560 | 434 | 31.1 | 26.1 | 26.1 |
| 3 | 27.1 | 479 | 434 | 24.1 | 26.1 | 26.1 |
| 3 | 28.1 | 299 | 435 | 33.1 | 40.1 | 40.1 |
| 3 | 29.1 | 474 | 347 | 1.1 | 4.1 | 4.1 |
| 3 | 30.1 | 235 | 222 | 25.2 | 8.2 | 8.2 |
| 3 | 31.1 | 542 | 390 | 18.1 | 21.1 | 21.1 |
| 3 | 32.1 | 329 | 226 | 3.1 | 20.1 | 20.1 |
| 3 | 33.1 | 333 | 435 | 28.1 | 13.1 | 13.1 |
| 3 | 34.1 | 239 | 257 | 2.1 | 8.3 | 8.3 |
| 3 | 35.1 | 545 | 227 | 6.1 | 36.1 | 36.1 |
| 3 | 36.1 | 530 | 272 | 38.1 | 22.1 | 22.1 |
| 3 | 36.2 | 585 | 338 | 22.1 | 21.1 | 21.1 |
| 3 | 37.1 | 242 | 433 | 39.1 | 28.1 | 28.1 |
| 3 | 37.2 | 587 | 260 | 6.1 | 35.1 | 35.1 |
| 3 | 38.1 | 521 | 306 | 5.1 | 22.1 | 22.1 |
| 3 | 39.1 | 224 | 381 | 12.3 | 14.1 | 14.1 |
| 3 | 40.1 | 289 | 388 | 42.2 | 14.1 | 14.1 |
| 3 | 41.1 | 387 | 274 | 42.4 | 11.5 | 11.5 |
| 3 | 42.1 | 292 | 309 | 23.1 | 11.2 | 11.2 |
| 3 | 42.2 | 286 | 352 | 14.1 | 40.1 | 40.1 |
| 3 | 42.3 | 335 | 363 | 9.2 | 43.1 | 43.1 |
| 3 | 42.4 | 407 | 301 | 11.5 | 14.2 | 14.2 |
| 3 | 43.1 | 373 | 387 | 11.6 | 42.3 | 42.3 |
| 4 | 1.1 | -12305 | -12030 | 27.1 | 27.2 | 27.2 |
| 4 | 1.2 | -12330 | -12066 | 29.1 | 18.1 | 18.1 |
| 4 | 2.1 | -12077 | -12191 | 34.1 | 10.1 | 10.1 |
| 4 | 3.1 | -12199 | -12213 | 17.1 | 33.1 | 33.1 |
| 4 | 4.1 | -12315 | -12127 | 16.1 | 5.1 | 5.1 |
| 4 | 5.1 | -12350 | -12148 | 38.1 | 36.2 | 36.2 |
| 4 | 8.1 | -12051 | -12184 | 25.2 | 10.1 | 10.1 |
| 4 | 8.2 | -12082 | -12173 | 12.5 | 12.6 | 12.6 |
| 4 | 8.3 | -12070 | -12201 | 12.8 | 2.1 | 2.1 |
| 4 | 9.1 | -12162 | -12086 | 42.1 | 11.3 | 11.3 |
| 4 | 9.2 | -12184 | -12140 | 11.4 | 11.3 | 11.3 |
| 4 | 10.1 | -12072 | -12180 | 25.2 | 2.1 | 2.1 |
| 4 | 11.1 | -12219 | -12084 | 43.1 | 42.3 | 42.3 |
| 4 | 11.2 | -12244 | -12130 | 42.3 | 14.3 | 14.3 |
| 4 | 11.3 | -12174 | -12130 | 9.2 | 11.4 | 11.4 |
| 4 | 11.4 | -12186 | -12148 | 9.2 | 11.3 | 11.3 |
| 4 | 11.5 | -12141 | -12161 | 23.1 | 11.6 | 11.6 |
| 4 | 11.6 | -12155 | -12169 | 11.5 | 23.1 | 23.1 |
| 4 | 12.1 | -12066 | -12060 | 39.1 | 14.2 | 14.2 |
| 4 | 12.2 | -12073 | -12126 | 15.1 | 25.1 | 25.1 |
| 4 | 12.3 | -12090 | -12155 | 12.5 | 25.1 | 25.1 |
| 4 | 12.4 | -12056 | -12162 | 25.1 | 8.1 | 8.1 |
| 4 | 12.5 | -12090 | -12167 | 8.2 | 12.3 | 12.3 |
| 4 | 12.6 | -12090 | -12182 | 34.1 | 8.2 | 8.2 |
| 4 | 12.7 | -12090 | -12199 | 34.1 | 2.1 | 2.1 |
| 4 | 12.8 | -12070 | -12211 | 8.3 | 2.1 | 2.1 |
| 4 | 13.1 | -12244 | -12000 | 27.1 | 43.1 | 43.1 |
| 4 | 14.1 | -12165 | -12000 | 24.1 | 28.1 | 28.1 |
| 4 | 14.2 | -12104 | -12066 | 37.3 | 39.1 | 39.1 |
| 4 | 14.3 | -12267 | -12148 | 11.2 | 41.1 | 41.1 |
| 4 | 15.1 | -12061 | -12128 | 12.2 | 25.1 | 25.1 |
| 4 | 16.1 | -12314 | -12164 | 4.1 | 5.1 | 5.1 |
| 4 | 16.2 | -12424 | -12146 | 22.1 | 36.1 | 36.1 |
| 4 | 17.1 | -12146 | -12211 | 32.1 | 11.6 | 11.6 |
| 4 | 18.1 | -12362 | -12043 | 31.1 | 1.2 | 1.2 |
| 4 | 19.1 | -12289 | -12214 | 33.1 | 16.1 | 16.1 |
| 4 | 21.1 | -12377 | -12081 | 31.1 | 18.1 | 18.1 |
| 4 | 22.1 | -12395 | -12134 | 38.1 | 16.2 | 16.2 |
| 4 | 23.1 | -12132 | -12158 | 11.5 | 11.6 | 11.6 |
| 4 | 23.2 | -12126 | -12116 | 42.2 | 42.1 | 42.1 |
| 4 | 24.1 | -12135 | -11999 | 28.1 | 14.1 | 14.1 |
| 4 | 25.1 | -12073 | -12158 | 12.3 | 12.4 | 12.4 |
| 4 | 25.2 | -12062 | -12185 | 10.1 | 8.1 | 8.1 |
| 4 | 26.1 | -12372 | -11999 | 18.1 | 31.1 | 31.1 |
| 4 | 27.1 | -12293 | -12017 | 1.1 | 27.2 | 27.2 |
| 4 | 27.2 | -12313 | -12002 | 27.1 | 1.1 | 1.1 |
| 4 | 28.1 | -12115 | -12000 | 37.2 | 24.1 | 24.1 |
| 4 | 29.1 | -12294 | -12066 | 1.2 | 1.1 | 1.1 |
| 4 | 30.1 | -12073 | -12232 | 12.8 | 8.3 | 8.3 |
| 4 | 31.1 | -12388 | -12047 | 18.1 | 21.1 | 21.1 |
| 4 | 32.1 | -12113 | -12211 | 12.7 | 17.1 | 17.1 |
| 4 | 33.1 | -12252 | -12213 | 19.1 | 41.1 | 41.1 |
| 4 | 34.1 | -12086 | -12191 | 2.1 | 12.7 | 12.7 |
| 4 | 35.1 | -12420 | -12213 | 44.1 | 36.2 | 36.2 |
| 4 | 44.1 | -12395 | -12213 | 35.1 | 36.2 | 36.2 |
| 4 | 36.1 | -12431 | -12102 | 16.2 | 22.1 | 22.1 |
| 4 | 36.2 | -12380 | -12167 | 5.1 | 22.1 | 22.1 |
| 4 | 37.1 | -12071 | -12010 | 37.2 | 28.1 | 28.1 |
| 4 | 37.2 | -12101 | -12004 | 28.1 | 37.1 | 37.1 |
| 4 | 37.3 | -12105 | -12049 | 14.2 | 40.1 | 40.1 |
| 4 | 38.1 | -12371 | -12132 | 22.1 | 5.1 | 5.1 |
| 4 | 39.1 | -12072 | -12070 | 12.1 | 14.2 | 14.2 |
| 4 | 40.1 | -12140 | -12047 | 37.3 | 42.1 | 42.1 |
| 4 | 41.1 | -12234 | -12165 | 42.3 | 14.3 | 14.3 |
| 4 | 42.1 | -12132 | -12083 | 9.1 | 14.2 | 14.2 |
| 4 | 42.2 | -12130 | -12126 | 23.2 | 23.1 | 23.1 |
| 4 | 42.3 | -12221 | -12134 | 11.2 | 41.1 | 41.1 |
| 4 | 43.1 | -12220 | -12048 | 11.1 | 13.1 | 13.1 |
| 5 | 1.1 | 295 | 418 | 29.1 | 29.2 | 29.2 |
| 5 | 1.2 | 334 | 387 | 29.2 | 14.4 | 14.4 |
| 5 | 1.3 | 257 | 383 | 29.1 | 41.1 | 41.1 |
| 5 | 2.1 | 233 | 288 | 34.1 | 25.3 | 25.3 |
| 5 | 3.1 | 399 | 261 | 46.1 | 32.1 | 32.1 |
| 5 | 4.1 | 384 | 388 | 5.1 | 14.3 | 14.3 |
| 5 | 5.1 | 378 | 399 | 14.3 | 4.1 | 4.1 |
| 5 | 5.2 | 375 | 329 | 38.3 | 22.1 | 22.1 |
| 5 | 6.1 | 573 | 303 | 23.2 | 11.8 | 11.8 |
| 5 | 8.1 | 215 | 297 | 39.2 | 25.2 | 25.2 |
| 5 | 8.2 | 245 | 275 | 25.3 | 10.1 | 10.1 |
| 5 | 9.1 | 565 | 447 | 11.3 | 11.2 | 11.2 |
| 5 | 10.1 | 227 | 278 | 25.3 | 25.2 | 25.2 |
| 5 | 11.1 | 435 | 474 | 24.1 | 40.1 | 40.1 |
| 5 | 11.2 | 551 | 487 | 9.1 | 11.3 | 11.3 |
| 5 | 11.3 | 571 | 429 | 9.1 | 26.1 | 26.1 |
| 5 | 11.4 | 499 | 426 | 27.3 | 42.2 | 42.2 |
| 5 | 11.5 | 480 | 408 | 27.3 | 27.2 | 27.2 |
| 5 | 11.6 | 570 | 359 | 23.3 | 11.7 | 11.7 |
| 5 | 11.7 | 583 | 353 | 23.3 | 11.8 | 11.8 |
| 5 | 11.8 | 588 | 339 | 11.7 | 23.3 | 23.3 |
| 5 | 12.1 | 217 | 417 | 39.1 | 1.3 | 1.3 |
| 5 | 12.2 | 227 | 359 | 15.1 | 1.3 | 1.3 |
| 5 | 12.3 | 245 | 313 | 14.5 | 34.1 | 34.1 |
| 5 | 13.1 | 329 | 473 | 14.1 | 40.1 | 40.1 |
| 5 | 14.1 | 285 | 472 | 28.1 | 13.1 | 13.1 |
| 5 | 14.2 | 357 | 409 | 14.3 | 14.4 | 14.4 |
| 5 | 14.3 | 366 | 397 | 5.1 | 14.2 | 14.2 |
| 5 | 14.4 | 343 | 397 | 1.2 | 29.2 | 29.2 |
| 5 | 14.5 | 258 | 326 | 41.2 | 12.3 | 12.3 |
| 5 | 45.1 | 449 | 369 | 27.2 | 26.3 | 26.3 |
| 5 | 15.1 | 219 | 334 | 25.1 | 12.2 | 12.2 |
| 5 | 16.1 | 402 | 347 | 5.2 | 22.1 | 22.1 |
| 5 | 17.1 | 306 | 262 | 19.1 | 33.1 | 33.1 |
| 5 | 18.1 | 519 | 342 | 38.2 | 36.2 | 36.2 |
| 5 | 19.1 | 275 | 265 | 8.2 | 17.1 | 17.1 |
| 5 | 19.2 | 231 | 237 | 30.1 | 8.2 | 8.2 |
| 5 | 21.1 | 314 | 336 | 36.1 | 36.6 | 36.6 |
| 5 | 46.1 | 393 | 290 | 22.1 | 3.1 | 3.1 |
| 5 | 22.1 | 394 | 309 | 46.1 | 5.2 | 5.2 |
| 5 | 47.1 | 338 | 368 | 1.2 | 29.2 | 29.2 |
| 5 | 23.1 | 484 | 480 | 42.1 | 11.1 | 11.1 |
| 5 | 23.2 | 557 | 312 | 6.1 | 11.8 | 11.8 |
| 5 | 23.3 | 572 | 357 | 11.6 | 11.7 | 11.7 |
| 5 | 24.1 | 419 | 468 | 11.1 | 40.1 | 40.1 |
| 5 | 25.1 | 226 | 310 | 8.1 | 12.3 | 12.3 |
| 5 | 25.2 | 221 | 287 | 39.2 | 10.1 | 10.1 |
| 5 | 25.3 | 237 | 277 | 8.2 | 10.1 | 10.1 |
| 5 | 26.1 | 541 | 412 | 42.2 | 11.3 | 11.3 |
| 5 | 26.2 | 513 | 390 | 38.1 | 27.3 | 27.3 |
| 5 | 26.3 | 429 | 394 | 26.5 | 26.4 | 26.4 |
| 5 | 26.4 | 456 | 412 | 27.1 | 26.5 | 26.5 |
| 5 | 26.5 | 456 | 407 | 26.4 | 27.1 | 27.1 |
| 5 | 27.1 | 459 | 412 | 26.4 | 26.5 | 26.5 |
| 5 | 27.2 | 465 | 395 | 26.5 | 27.1 | 27.1 |
| 5 | 27.3 | 487 | 406 | 11.5 | 38.1 | 38.1 |
| 5 | 28.1 | 266 | 474 | 14.1 | 37.1 | 37.1 |
| 5 | 29.1 | 291 | 384 | 41.1 | 36.3 | 36.3 |
| 5 | 29.2 | 332 | 387 | 1.2 | 14.4 | 14.4 |
| 5 | 30.1 | 224 | 257 | 10.1 | 19.2 | 19.2 |
| 5 | 31.1 | 470 | 337 | 38.2 | 45.1 | 45.1 |
| 5 | 32.1 | 447 | 265 | 3.1 | 36.4 | 36.4 |
| 5 | 33.1 | 342 | 262 | 17.1 | 36.5 | 36.5 |
| 5 | 34.1 | 241 | 294 | 2.1 | 25.3 | 25.3 |
| 5 | 35.1 | 543 | 265 | 6.1 | 23.2 | 23.2 |
| 5 | 36.1 | 298 | 358 | 36.3 | 41.1 | 41.1 |
| 5 | 36.2 | 510 | 317 | 18.1 | 38.2 | 38.2 |
| 5 | 36.3 | 287 | 360 | 41.1 | 36.1 | 36.1 |
| 5 | 36.4 | 425 | 313 | 22.1 | 46.1 | 46.1 |
| 5 | 36.5 | 357 | 308 | 38.3 | 5.2 | 5.2 |
| 5 | 36.6 | 311 | 306 | 21.1 | 17.1 | 17.1 |
| 5 | 37.1 | 231 | 472 | 28.1 | 14.1 | 14.1 |
| 5 | 38.1 | 492 | 384 | 26.2 | 27.3 | 27.3 |
| 5 | 38.2 | 497 | 350 | 18.1 | 31.1 | 31.1 |
| 5 | 38.3 | 355 | 332 | 5.2 | 36.5 | 36.5 |
| 5 | 39.1 | 224 | 416 | 12.1 | 1.3 | 1.3 |
| 5 | 39.2 | 211 | 288 | 8.1 | 25.2 | 25.2 |
| 5 | 40.1 | 392 | 473 | 24.1 | 11.1 | 11.1 |
| 5 | 41.1 | 288 | 364 | 36.3 | 36.1 | 36.1 |
| 5 | 41.2 | 274 | 330 | 14.5 | 36.3 | 36.3 |
| 5 | 42.1 | 487 | 472 | 23.1 | 11.4 | 11.4 |
| 5 | 42.2 | 523 | 420 | 26.1 | 11.4 | 11.4 |
| 5 | 43.1 | 396 | 427 | 5.1 | 4.1 | 4.1 |
+-------+-------+--------+--------+------+------+------+