# HTTP(S) load balancing with Ingress ## Resources Features of GKE Ingress from the Google Cloud docs: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress It does hostname-aware HTTP(S) load balancing, and is billed like a regular Load Balancer (https://cloud.google.com/compute/pricing#lb). The advantages are that we can use one set of firewall rules (ports 80 and 443) for multiple services, and easy Let's Encrypt certificates for services with no built-in support for it This 3 part article was a good resource: https://medium.com/google-cloud/global-kubernetes-in-3-steps-on-gcp-8a3585ec8547 https://medium.com/google-cloud/global-ingress-in-practice-on-google-container-engine-part-1-discussion-ccc1e5b27bd0 https://medium.com/google-cloud/global-ingress-in-practice-on-google-container-engine-part-2-demo-cf587765702 I couldn't find information about setting `ingress.kubernetes.io/rewrite-target` to `/` anywhere else, without it only `/` worked on an host, all other URLs would go to the default backend and return a 404. cert-manager, for automated (among others) Let's Encrypt certificates: https://docs.cert-manager.io/en/release-0.8/ ## Create a global IP Ephemeral IPs are only regional, and you lose them if you have to recreate the Ingress gcloud compute addresses create ingress-ip --global ## Create the ingress A ClusterIP will not work, because it is allocating random ports. Explicitly create a NodePort to expose your service. On GKE, health checks are configured automatically cat < test-server-nodeport.yaml apiVersion: v1 kind: Service metadata: name: test-server-nodeport spec: ports: - name: http port: 80 targetPort: 3000 type: NodePort selector: name: test-server EOF kubectl apply -f test-server-nodeport.yaml Create the ingress resource cat < ingress-main.yaml # A GCE Ingress that uses cert-manager to manage Let's Encrypt certificates apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-main annotations: # Required, otherwise only the / path works # https://medium.com/google-cloud/global-ingress-in-practice-on-google-container-engine-part-1-discussion-ccc1e5b27bd0 ingress.kubernetes.io/rewrite-target: / certmanager.k8s.io/cluster-issuer: "letsencrypt-production" certmanager.k8s.io/acme-challenge-type: http01 # Created using the following command # gcloud compute addresses create ingress-ip --global kubernetes.io/ingress.global-static-ip-name: "ingress-ip" spec: tls: - hosts: - test.kosmos.org secretName: test-kosmos-org-cert - test2.kosmos.org secretName: test2-kosmos-org-cert rules: - host: test.kosmos.org http: paths: - backend: serviceName: test-server-nodeport servicePort: 80 - host: test2.kosmos.org http: paths: - backend: serviceName: test-server-nodeport servicePort: 80 EOF kubectl apply -f ingress-main.yaml ## cert-manager ### Create the cert-manager resources cert-manager provides a Let's Encrypt certificate issuer, and lets you mount it in an Ingress resource, making it possible to use HTTP ACME challenges Get the reserved IP you created in the first step: $ gcloud compute addresses list --global NAME ADDRESS/RANGE TYPE PURPOSE NETWORK REGION SUBNET STATUS ingress-ip 35.244.164.133 EXTERNAL IN_USE Set the DNS record for the domain you want a Let's Encrypt cert for to this IP. Now it's time to create the cert-manager https://docs.cert-manager.io/en/release-0.8/getting-started/install/kubernetes.html kubectl create namespace cert-manager kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.8.1/cert-manager.yaml --validate=false I had to run the apply command twice for it to create all the resources. On the first run I got these errors. Running it a second time successfully created all the resources unable to recognize "cert-manager.yaml": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1" unable to recognize "cert-manager.yaml": no matches for kind "Certificate" in version "certmanager.k8s.io/v1alpha1" unable to recognize "cert-manager.yaml": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1" unable to recognize "cert-manager.yaml": no matches for kind "Certificate" in version "certmanager.k8s.io/v1alpha1" We name the ingress explicitely so it only runs on one. Having only one IP to set on the DNS records makes the HTTP validation easier. Using the class would attach the validation endpoint to all Ingresses of that class (https://docs.cert-manager.io/en/latest/reference/api-docs/index.html#acmechallengesolverhttp01ingress-v1alpha1) cat < letsencrypt-staging.yaml apiVersion: certmanager.k8s.io/v1alpha1 kind: ClusterIssuer metadata: name: letsencrypt-staging spec: acme: # Let's Encrypt will use this to contact you about expiring # certificates, and issues related to your account. email: ops@kosmos.org server: https://acme-staging-v02.api.letsencrypt.org/directory privateKeySecretRef: # Secret resource used to store the account's private key. name: letsencrypt-staging-account-key solvers: - http01: ingress: name: ingress-main EOF cat < letsencrypt-production.yaml apiVersion: certmanager.k8s.io/v1alpha1 kind: ClusterIssuer metadata: name: letsencrypt-production spec: acme: # Let's Encrypt will use this to contact you about expiring # certificates, and issues related to your account. email: ops@kosmos.org server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: # Secret resource used to store the account's private key. name: letsencrypt-production-account-key solvers: - http01: ingress: name: ingress-main ## Add another service To add another service behind the Ingress, you set the DNS entry for its domain to the Ingress IP, deploy your service, create a NodePort to expose it, and finally add its host to the Ingress config (both tls and rules, see example above)