Regarding the properties of k-nearest neighbors, on page 38 of Elements of Statistical Learning, the authors write:
"...as the dimension p gets large, so does the metric size of the k-nearest neighborhood. So settling for nearest neighborhood as a surrogate for conditioning will fail us miserably."
Does this mean that, holding k constant, as we add features to a model, the distance between outcomes and thus the size of neighborhoods increases, so the model's variance increases?
The curse of dimensionality comes in various shapes. Especially for machine learning, there is a discussion here.
Generally, with increasing dimensionality, the relative difference in distances between points becomes increasingly small. For d=1000 dimensions, it is highly unlikely, that any point A in a random dataset is significantly closer to a given point B than any other point. In a way this can be explained by saying that with d=1000 it is very unlikely that a point A is closer to a point B in the vast majority of dimensions (at least unlikely to be closer than any other arbitrary point).
Another aspect is that the volumetric properties become unintuitive for increasing 'd'. For example, even when assuming a relatively moderate d=25 (if I remember correctly), the volume of a the unit-cube (length of edge = 1) is 1,000,000 bigger than the volume of the unit-sphere (sphere with diameter = 1). I mention this because your quote mentions 'metric size', but I'm not sure how this affects kNN.