Skip Unreachable

[awm-achi]

Is there a way to have the plan skip unreachable nodes? Currently, if a node is unreachable, since we require concurrency = 1, it appears to block any of the remaining upgrades from executing. I understand that this should typically be handled by taints, but we have a diverse range of taints that are utilized, and there is no way to specify tolerances for all except unreachable taints to my knowledge. Our current approach involves setting up a pre-sync hook to label nodes as unreachable immediately before a plan runs. However, I am curious if this is a better solution to address this issue.

[brandond]

No... this is not currently possible and not planned. I'm not sure why you'd want the plan to skip upgrading a node just because it's offline when the plan is executed?

I could see perhaps sorting the node list so that unreachable nodes were done last to avoid blocking other nodes, but I wouldn't want to consider the plan complete if some nodes were offline and didn't get upgraded.

[awm-achi]

Sorting would work too, but I don't think you can do that currently?

[brandond]

No, but that would be a good enhancement if you wanted to open a PR.

You might be the first person to run into this. I'm not sure a lot of folks regularly operate their clusters with nodes unreachable for an extended period of time.