diff --git a/doc/ingress.md b/doc/ingress.md new file mode 100644 index 0000000..490c0ef --- /dev/null +++ b/doc/ingress.md @@ -0,0 +1,180 @@ +# HTTP(S) load balancing with Ingress + +## Resources + +Features of GKE Ingress from the Google Cloud docs: +https://cloud.google.com/kubernetes-engine/docs/concepts/ingress + +It does hostname-aware HTTP(S) load balancing, and is billed like a regular +Load Balancer (https://cloud.google.com/compute/pricing#lb). The advantages are +that we can use one set of firewall rules (ports 80 and 443) for multiple +services, and easy Let's Encrypt certificates for services with no built-in +support for it + +This 3 part article was a good resource: + +https://medium.com/google-cloud/global-kubernetes-in-3-steps-on-gcp-8a3585ec8547 +https://medium.com/google-cloud/global-ingress-in-practice-on-google-container-engine-part-1-discussion-ccc1e5b27bd0 +https://medium.com/google-cloud/global-ingress-in-practice-on-google-container-engine-part-2-demo-cf587765702 + +I couldn't find information about setting +`ingress.kubernetes.io/rewrite-target` to `/` anywhere else, without it only +`/` worked on an host, all other URLs would go to the default backend and +return a 404. + +cert-manager, for automated (among others) Let's Encrypt certificates: +https://docs.cert-manager.io/en/release-0.8/ + +## Create a global IP + +Ephemeral IPs are only regional, and you lose them if you have to recreate the +Ingress + + gcloud compute addresses create ingress-ip --global + +## Create the ingress + +A ClusterIP will not work, because it is allocating random ports. Explicitly +create a NodePort to expose your service. On GKE, health checks are configured +automatically + + cat < test-server-nodeport.yaml + apiVersion: v1 + kind: Service + metadata: + name: test-server-nodeport + spec: + ports: + - name: http + port: 80 + targetPort: 3000 + type: NodePort + selector: + name: test-server + EOF + kubectl apply -f test-server-nodeport.yaml + +Create the ingress resource + + cat < ingress-main.yaml + # A GCE Ingress that uses cert-manager to manage Let's Encrypt certificates + apiVersion: extensions/v1beta1 + kind: Ingress + metadata: + name: gitea-ingress + annotations: + # Required, otherwise only the / path works + # https://medium.com/google-cloud/global-ingress-in-practice-on-google-container-engine-part-1-discussion-ccc1e5b27bd0 + ingress.kubernetes.io/rewrite-target: / + certmanager.k8s.io/cluster-issuer: "letsencrypt-production" + certmanager.k8s.io/acme-challenge-type: http01 + # Created using the following command + # gcloud compute addresses create ingress-ip --global + kubernetes.io/ingress.global-static-ip-name: "ingress-ip" + spec: + tls: + - hosts: + - test.kosmos.org + secretName: test-kosmos-org-cert + - test2.kosmos.org + secretName: test2-kosmos-org-cert + rules: + - host: test.kosmos.org + http: + paths: + - backend: + serviceName: test-server-nodeport + servicePort: 80 + - host: test2.kosmos.org + http: + paths: + - backend: + serviceName: test-server-nodeport + servicePort: 80 + EOF + kubectl apply -f ingress-main.yaml + + +## cert-manager + +### Create the cert-manager resources + +cert-manager provides a Let's Encrypt certificate issuer, and lets you mount it +in an Ingress resource, making it possible to use HTTP ACME challenges + +Get the reserved IP you created in the first step: + + $ gcloud compute addresses list --global + NAME ADDRESS/RANGE TYPE PURPOSE NETWORK REGION SUBNET STATUS + ingress-ip 35.244.164.133 EXTERNAL IN_USE + +Set the DNS record for the domain you want a Let's Encrypt cert for to this IP. + +Now it's time to create the cert-manager + +https://docs.cert-manager.io/en/release-0.8/getting-started/install/kubernetes.html + + kubectl create namespace cert-manager + + kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.8.1/cert-manager.yaml --validate=false + +I had to run the apply command twice for it to create all the resources. On the +first run I got these errors. Running it a second time successfully created all +the resources + + unable to recognize "cert-manager.yaml": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1" + unable to recognize "cert-manager.yaml": no matches for kind "Certificate" in version "certmanager.k8s.io/v1alpha1" + unable to recognize "cert-manager.yaml": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1" + unable to recognize "cert-manager.yaml": no matches for kind "Certificate" in version "certmanager.k8s.io/v1alpha1" + +We name the ingress explicitely so it only runs on one. Having only one IP to +set on the DNS records makes the HTTP validation easier. Using the class would +attach the validation endpoint to all Ingresses of that class +(https://docs.cert-manager.io/en/latest/reference/api-docs/index.html#acmechallengesolverhttp01ingress-v1alpha1) + + cat < letsencrypt-staging.yaml + apiVersion: certmanager.k8s.io/v1alpha1 + kind: ClusterIssuer + metadata: + name: letsencrypt-staging + spec: + acme: + # Let's Encrypt will use this to contact you about expiring + # certificates, and issues related to your account. + email: ops@kosmos.org + server: https://acme-staging-v02.api.letsencrypt.org/directory + privateKeySecretRef: + # Secret resource used to store the account's private key. + name: letsencrypt-staging-account-key + solvers: + - http01: + ingress: + name: gitea-ingress + EOF + + cat < letsencrypt-production.yaml + apiVersion: certmanager.k8s.io/v1alpha1 + kind: ClusterIssuer + metadata: + name: letsencrypt-production + spec: + acme: + # Let's Encrypt will use this to contact you about expiring + # certificates, and issues related to your account. + email: ops@kosmos.org + server: https://acme-v02.api.letsencrypt.org/directory + privateKeySecretRef: + # Secret resource used to store the account's private key. + name: letsencrypt-production-account-key + solvers: + - http01: + ingress: + name: gitea-ingress + + +## Add another service + +To add another service behind the Ingress, you set the DNS entry for its domain +to the Ingress IP, deploy your service, create a NodePort to expose it, and +finally add its host to the Ingress config (both tls and rules, see example +above)