I'm guessing the answer is going to center around the word "complexity" cited in their original comment. That is: I would guess it's YAGNI more than EKS itself
Ah. Yeah I'm setting things up with managed node groups and it doesn't seem so bad so far... Waiting for the other shoe to drop after so much doom saying though haha. Luckily we removed the need for anything stateful, so I can ignore the EBS-CSI shortcomings. Also trying to keep it simple/minimal when it comes to ingress and networking.
Cool. I've never had any issue with EBS CSI driver itself, the biggest issue were idiosyncracies of EBS itself, like the mount count limit or availability zone requirements. These need ugly workarounds, like limiting your volumes to a single AZ, so, no HA.
On the other side, their VPC CNI plugin and their ingress controller are pretty much set and forget.
Yeah basically it is that EBS limitation (AZ-specific) combined with autoscaling causing quirky failure modes unless you are careful about how you set things up.
We used AWS EKS in the old days and we never liked the extreme complexity of it.
With two Spring Boot apps, a database and Redis running across Ubuntu servers, we found simpler tools to distribute and scale workloads.
Since compute is dirt cheap, we over-provision and sleep well.
We have live alerts and quarterly reviews (just looking at a dashboard!) to assess if we balance things well.
K8s on EKS was not pleasant, I wanna make sure I never learn how much worse it can get across European VPS providers.