如何在 EKS 部署 Nginx Ingress Controller

花了一點時間研究了如何將 Nginx Ingress Controller 部署到 EKS 叢集當中。首先,我們可以決定要怎麼使用 Nginx Ingress Controller,你有幾種不同的選擇:

  1. CLB or NLB:將控制器部署為 Classic Load Balancer 或 Network Load Balancer 類別
  2. Public or Private:負載均衡器是可公開存取的或是僅能私有存取(只有位於你的 VPC 網路內才能夠存取)
  3. HTTP or HTTPS:是否要配置 SSL,讓客戶端到 Load Balancer 之間有加密連線

選擇 CLB 或是 NLB

首先,先確定要使用的是 Classic Load Balancer 還是 Network Load Balancer,從安裝上來說,當叢集裡面存在 AWS Load Balancer Controller 時,你的 Nginx Ingress Controller 會被建立成一個 Network Load Balancer(NLB)。若 AWS Load Balancer Controller 不存在,則會把該控制器部署為 Classic Load Balancer Controller(CLB)。由於 CLB 被 AWS 官方指南稱為上一代的負載平衡器 [1],因此已經不推薦使用。

Classic Load Balancer is the Elastic Load Balancing previous-generation load balancer. It supports routing HTTP, HTTPS, or TCP request traffic to different ports on environment instances.

你可以使用 kubectl -n ingress-nginx describe svc ingress-nginx-controller 來檢查已部署的控制器是屬於 Classic Load Balancer 還是 Network Load Balancer:

1
2
3
4
5
# Classic Load Balancer 的網址範例
a878e058e56d0458ba59f88294c97283-e17eb490e17ec9e3.elb.eu-west-1.amazonaws.com

# Network Load Balancer 的網址範例,會使以 k8s-ingressn-ingressn- 作為開頭
k8s-ingressn-ingressn-e78cc5a707-15a0244dd914ab91.elb.eu-west-1.amazonaws.com

選擇可公開存取或僅能私有存取

在預設情況之下,AWS 負載平衡器預設為僅限私有存取,也就是說,只有與你相同 VPC 的其他機器(如:EC2 Instances)能夠存取這個負載平衡器的 endpoint。如果你想要安裝私有存取的負載平衡器,你可以直接按照 Nginx Ingress Controller 文件中所述,執行以下的指令來進行安裝(請注意:以下命令的安裝版本為 v1.8.2,可能不是最新版本):

1
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/aws/deploy.yaml

不過,如果你想要安裝的是可以提供公開存取能力的負載平衡器,你則需要修改其內容:

1
2
curl -O https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/aws/deploy.yaml
vim deploy.yaml

deploy.yaml 檔案之中找到這一段,並加上 annotation service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing",這會讓 AWS 配置此負載平衡器為可公開存取,你也可以從 Annotations - AWS Load Balancer Controller 找到其他能夠設定的負載平衡器參數:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 找到 Service ingress-nginx-controller 的定義
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
service.beta.kubernetes.io/aws-load-balancer-type: nlb
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing" # <---- 加上這一行內容
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.8.2
name: ingress-nginx-controller
namespace: ingress-nginx

配置 TLS 相關的設定

如果要進一步為負載平衡器配置 TLS,則需要參考 Nginx Ingress Controller 文件中,將 deploy.yaml 下載,並修改其中的內容。

1
curl -O https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/aws/nlb-with-tls-termination/deploy.yaml
  1. proxy-real-ip-cidr 改成 EKS cluster 使用的 CIDR
  2. service.beta.kubernetes.io/aws-load-balancer-ssl-cert 當中的值填入要使用的憑證 ARN。

部署範例服務

我們使用 Nginx 官方提供的 example 來進行改寫:

  • cafe-ing.yaml:定義了 Ingress 物件。
  • cafe-dep.yaml:定義了 teacoffee 兩個不同 Service,各對應到一個 Pod。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
# cafe-ing.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: coffee
spec:
replicas: 1
selector:
matchLabels:
app: coffee
template:
metadata:
labels:
app: coffee
spec:
containers:
- name: coffee
image: nginxdemos/nginx-hello:plain-text
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: coffee-svc
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: coffee
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tea
spec:
replicas: 1
selector:
matchLabels:
app: tea
template:
metadata:
labels:
app: tea
spec:
containers:
- name: tea
image: nginxdemos/nginx-hello:plain-text
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: tea-svc
labels:
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: tea
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# cafe-ing.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: cafe-ingress
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /tea
pathType: Prefix
backend:
service:
name: tea-svc
port:
number: 80
- path: /coffee
pathType: Prefix
backend:
service:
name: coffee-svc
port:
number: 80

進行測試

當部署好之後,我們能夠在我們的 EKS 叢集裡面確認 Service 與 Deployment 存在:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
coffee-svc ClusterIP 10.100.141.198 <none> 80/TCP 5m41s
kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 39d
tea-svc ClusterIP 10.100.80.247 <none> 80/TCP 5m41s

$ kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
coffee 1/1 1 1 6m
tea 1/1 1 1 6m

$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coffee-7dd75bc79b-mhtdv 1/1 Running 0 9m8s 192.168.34.195 ip-192-168-39-19.eu-west-1.compute.internal <none> <none>
tea-df5655878-9vrp6 1/1 Running 0 9m8s 192.168.61.129 ip-192-168-39-19.eu-west-1.compute.internal <none> <none>

$ kubectl describe ingress cafe-ingress
Name: cafe-ingress
Labels: <none>
Namespace: default
Address: k8s-ingressn-ingressn-e78cc5a707-15a0244dd914ab91.elb.eu-west-1.amazonaws.com
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
*
/tea tea-svc:80 (192.168.61.129:8080) # <--- 可以看到對於 /tea 與 /coffee 會把請求轉送到不同的 Service endpoint
/coffee coffee-svc:80 (192.168.34.195:8080)
Annotations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 7m14s (x2 over 7m27s) nginx-ingress-controller Scheduled for sync

嘗試訪問 Nginx Ingress Controller 的 URL:

1
2
3
4
5
6
7
8
9
10
11
12
13
$ curl k8s-ingressn-ingressn-e78cc5a707-15a0244dd914ab91.elb.eu-west-1.amazonaws.com/tea
Server address: 192.168.61.129:8080
Server name: tea-df5655878-9vrp6
Date: 27/Jan/2024:13:41:39 +0000
URI: /tea
Request ID: dd1c382348657763b2d50e565bd87839

$ curl k8s-ingressn-ingressn-e78cc5a707-15a0244dd914ab91.elb.eu-west-1.amazonaws.com/coffee
Server address: 192.168.34.195:8080
Server name: coffee-7dd75bc79b-mhtdv
Date: 27/Jan/2024:13:41:45 +0000
URI: /coffee
Request ID: fa9771896557c49d783955a6fe4b7e25

如何在 EKS 部署 Nginx Ingress Controller

https://weiyu.dev/tw/nginx-ingress-controller-in-eks/

作者

Wei-Yu Chen

發表於

2024-01-27

更新於

2024-05-06

許可協議

評論