DevOps & cloud

Lately I have been hitting situations where containers overload the system disk on my EKS nodes. Once the root disk maxes out its throughput, the operating system freezes, CPU usage spikes, and the node drops into NotReady. Even worse, the node stops accepting SSH, so I cannot inspect what actually went wrong. Kubernetes will eventually reschedule the Pods, but only after the node has stayed in NotReady for five minutes.

To keep this from happening again, I split the system disk and the container data disk into two separate EBS volumes so that container workloads always land on the second volume.

Within Amazon EKS, we use IAM Role for Service Account (IRSA) or EKS Pod Identity to grant applications permission to call AWS APIs. While Pod Identity Agent won't be covered in this article, we'll explore how IRSA works in your EKS cluster. Understanding this feature better will help you troubleshoot issues when they arise.

Applications running in your cluster may need to call AWS APIs to manage AWS resources. For example, when you install an EKS Add-on like the EBS CSI Controller, it needs to interact with AWS APIs to perform tasks such as mounting EBS volumes to EKS worker nodes. The kubelet can then mount these volumes into Pods via the Container Storage Interface.

I spent some time researching how to deploy the Nginx Ingress Controller to an EKS cluster. Firstly, we can decide how to use the Nginx Ingress Controller, and you have several different choices:

Firstly, determine whether to use a Classic Load Balancer or a Network Load Balancer. In terms of installation, when an AWS Load Balancer Controller exists within the cluster, your Nginx Ingress Controller will be created as a Network Load Balancer (NLB). If the AWS Load Balancer Controller does not exist, then the controller will be deployed as a Classic Load Balancer Controller (CLB). As CLB is referred to as a previous generation load balancer by the AWS official guide [1], it is no longer recommended for use.

This article primarily introduces the core operational process of Karpenter, including how Karpenter filters Pods that need to be scheduled and the actual process of scheduling Pods onto worker nodes.

Karpenter is a tool designed by AWS for Kubernetes clusters. It scales the cluster based on the workload of worker nodes (instances). Karpenter has several features and functions:

Once CodePipeline and CodeBuild are connected, a new commit can trigger a build and place the generated files in S3. There are still several pieces needed for a comfortable browsing experience: a domain name, HTTPS, delivery through CloudFront, and working directory-style links.

This part explains the original S3 and CloudFront setup, including the small Lambda@Edge function used to rewrite requests for nested index pages.

With CodePipeline configured, the source retrieved from GitHub appears in the artifact bucket as a ZIP archive. The next step is to tell CodeBuild how to turn that source into the website. Those instructions belong in buildspec.yml.

A buildspec describes the commands used to build and test a project, much like a Jenkins Pipeline script. CodeBuild normally looks for buildspec.yml in the source root:

After working with CI/CD, I came across AWS CodePipeline and had a simple idea: when I commit a new article to GitHub, why not let the site rebuild and publish itself? Hosting the generated files on S3 would also remove the need to maintain a VPS.

That small idea eventually grew into a pipeline involving CodePipeline, CodeBuild, S3, Route 53, CloudFront, Lambda, and IAM. This first part explains how the pipeline fits together, how to create it, and why its service roles need access to the artifact bucket.