Squeezing more (orange) juice from Kubernetes

We attended a lot of interesting sessions – too many to list them all here. Again, my personal bias is for anything related to scalability, performance and optimization. 

A very interesting speech that stood out for me was the one from Carlos Sanchez, Cloud Engineer at Adobe, who described how they use standard Kubernetes capabilities to build custom solutions in order to minimize infrastructure resource waste while avoiding effort from their dev teams. 

To bring down costs, Adobe leverages Kubernetes autoscalers extensively. The biggest benefits were achieved by autoscaling clusters and pods. They achieved 30-50% savings by using HPA (horizontal pod autoscaler), but making it work correctly is not easy (CPU autoscaling being problematic, while throughput-based being better suited). Vertical pod autoscaler (VPA) only achieved 5-15% savings, but was only used in development as not considered suitable for prod environments. Moreover, they also declared 60-80% savings by hibernating development environments.

Of course, one of the main challenges are optimizing Java workloads: the JVM (or any other runtime, for that matter) is not playing nice with Kubernetes autoscaling mechanisms – not to mention the startup issues. Their recommendation was not to stick to JVM default settings, and to ensure that GC type and memory heap are set according to configured resources [on this topic I would recommend the Dynatrace PurePerformance podcast with our CTO].

Another interesting talk was delivered by Vincent Sevel, Architect /Platform Ops at Lombard Odier, who described the challenges of managing large-scale, dynamic environments where simply there is no time to fine-tune tens of workloads running on thousands of pods when dozens of releases are brought in production each day.  

As for Adobe, also Lombard Odier goal is to avoid any resource waste without sacrificing SLOs and avoiding workload to crash, and also that would not require actions from dev teams. Indeed the main problem is that dev teams are often raising CPU requests in the hope to fix issues so CPUs are always insufficient, clusters are too small and resources are overallocated [on this I would recommend our recent blog].

They also described several problems that we are very familiar with (and helped our customers to address), such as the JVM sizing dilemma, how to avoid startup problems while not wasting cluster stability due to CPU throttling [watch this Is it observable episode], and the well-known challenge of balancing performance vs cost.

Finally, I would like to mention the session from Moviri, Akamas strategic partner. I may be biased, but the session delivered by Mauro Pessina, Manager at Moviri Performance Engineering, was very well received with more than 700 attendees (see pictures here below).