Validating that Cluster Costs are Justified

Which results in an efficiency (cost-weighted utilization) of around 10% for the last 7 days.

13. You can use the other periods to review historical data and identify usage patterns, peak loads, and times of underutilization to gain more insight.
14. By reviewing the charts, we can see that workloads associated with this deployment are probably well provisioned on memory but are possibly overprovisioned on CPU; however, as the cost of CPU is significantly higher than the cost of memory, we have quickly identified a possible reason why the cost of this deployment is high relative to its utilization.
15. Further investigation is likely warranted to confirm that the CPU is overprovisioned, but within a minute or less, you have valuable insights into why this cluster is the most expensive cluster in your environment.
16. You can examine other tabs in the Information Panel on the deployment report view to gain additional insights.
17. If the analysis of the deployment identifies an issue of interest, click the deployment name link in the Information Panel to view the deployment in the visualization:
    
    into
18. In the visualization, double-click the deployment (it will be highlighted) to drill into its workloads and change the perspective selector from Health to Cost to see which workloads might need more analysis:
    

In the pod visualization, we can see that all of the pods are under cost pressure.

19. By reviewing the metrics for just one pod in this deployment, we can see that CPU usage has short bursts of very high spikes (one spike each day of ~600%), but otherwise usage is around 10%:
    
20. This could indicate that we are spending more than necessary to accommodate the spikes and that there are opportunities for cost savings if, for example, any of the following might be options:
21. • breaking the work performed during the spikes into smaller tasks that could be distributed more evenly throughout the day
    • scaling resources dynamically using Kubernetes Horizontal Pod Autoscaling (HPA) to automatically accommodate increased demand during peak periods
    • right-sizing resource requests and limits for the pod based on its actual resource requirements during peak and non-peak periods
    • conducting thorough performance testing and profiling to identify any inefficiencies or bottlenecks within the application
21. If the analysis of the deployments did not identify any significant discrepancy between cost and utilization, continue your investigation by using the
    
    in the Reports > Total Cost > Deployments breadcrumb to select another object type of interest.
22. Typically, reviewing objects in the following order will quickly identify discrepancies between cost and utilization that might be contributing to higher-than-expected costs:
23. • ReplicaSets
      Note: You can also double-click the deployment row of most interest to list the replicasets for the selected cluster, namespace, and deployment.
    • StatefulSets
    • DaemonSets
23. If, after reviewing the charts and the Information Panel tabs for these objects as you did for deployments, the utilization of each object is within an acceptable range, a review of the nodes might determine if the high cluster costs are related to over provisioned nodes.
24. In the breadcrumb, click Namespaces and then use the
    
    to select Nodes.
25. Repeat the cost-to-utilization analysis for the highest cost node(s) in the cluster.
26. If node utilization is within an acceptable range, then the cluster costs can be justified, no immediate action is needed, and monitoring should continue.
27. If node utilization is not within an acceptable range, you might have too many nodes running or scheduling constraints that prevent pods from being scheduled to a node; consider the following:
28. • Review the scaling policies configured for your cluster and, if necessary, adjust the scaling thresholds and policies to better match the workload requirements
    • Review the distribution of workloads across the nodes in the cluster and consider rebalancing workloads or adjusting pod scheduling and affinity configurations to distribute workloads more evenly across nodes
    • Evaluate the resource requests and limits set for pods running in your cluster and adjust resource requests and limits as needed to prevent overprovisioning of resources and to optimize resource utilization across nodes