Nginx-lego

This chart bootstraps an nginx-lego deployment on a Kubernetes cluster using the Helm package manager. It is container, which provides nginx based https/ssl proxy. It uses https://letsencrypt.org/ and https://github.com/xenolf/lego to automatically obtain and renew certificates.

It is mainly intended for use with internal (not publicly accessible) services, such as internal Docker registries, so it uses DNS challenge and API enabled DNS servers.

Prerequisites

• Kubernetes 1.4+ with Beta APIs enabled

Nginx

nginx [engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server, originally written by Igor Sysoev. For a long time, it has been running on many heavily loaded Russian sites including Yandex, Mail.Ru, VK, and Rambler. According to Netcraft, nginx served or proxied 25.03% busiest sites in August 2018. Here are some of the success stories: Dropbox, Netflix, Wordpress.com, FastMail.FM.

The sources and documentation are distributed under the 2-clause BSD-like license.

HTTPS ingress with nginx + kube-lego

Kubernetes Ingress Objects are used to manage HTTP(S) access from the internet to inside the Kubernetes cluster. Among other things, it lets us do the following:

• Provide a HTTPS end point so users can connect to us securely
• Direct traffic to various Services based on hostnames or URL paths
• Allow using one public IP address for multiple domain names
• We use the nginx-ingress provider to handle our Ingress needs.

Nginx Ingress

We run on Google Cloud’s Kubernetes Engine. Even though GKE comes pre-installed with the Google Cloud Load Balancer Ingress provider, we decided to use nginx instead for the following reasons:

• GCLB has a 30s default timeout on all HTTP connections. This is counted not just when the connection is idle but from connection start time. This is particularly bad for WebSockets since those connections usually last for a lot longer than 30s! There is no easy way to configure this timeout from inside Kubernetes.
• GCLB does not guarantee you can use the same IP for multiple domains. We want the various subdomains of mybinder.org to point to the same IP so we can easily add/remove new services without waiting for DNS propagation delay.

Configuration with Ingress objects

Ingress objects are used to tell the ingress controllers which requests should be routed to which Service objects. Usually, the rules either check for a hostname (like mybinder.org or prometheus.mybinder.org) and/or a URL prefix (like /metrics or /docs). You can see all the ingress objects present with kubectl --namespace=prod get ingress.

The following ingress objects currently exist:

• jupyterhub - Directs traffic to hub.mybinder.org. The zero-to-jupyterhub guide has more documentation.
• binderhub - Directs traffic to mybinder.org. You can find more details about this in the binderhub helm chart.
• redirector - Directs traffic to the HTTP redirector we run for mybinder.org. This helps do redirects such as docs.mybinder.org or beta.mybinder.org. The list of redirects is configured in mybinder/values.yaml. The code for this is in mybinder/templates/redirector in this repo.
• static - Directs traffic into static.mybinder.org. We serve the mybinder.org badges from a different domain for privacy reasons. This ingress lets us direct traffic only from static.mybinder.org/badge.svg to the binder pod.
• prometheus-server - Directs traffic to prometheus.mybinder.org. Configured under prometheus in both mybinder/values.yaml and config/prod.yaml.
• grafana - Directs traffic to grafana.mybinder.org. Configured under grafana in both mybinder/values.yaml and config/prod.yaml.

kube-lego-nginx - Used by kube-lego for doing automatic HTTPS certificate renewals.

Lego Features

Register with CA

Obtain certificates, both from scratch or with an existing CSR

Renew certificates

Revoke certificates

Robust implementation of all ACME challenges

• HTTP (http-01)
• DNS (dns-01)
• TLS (tls-alpn-01)
• SAN certificate support
• Comes with multiple optional DNS providers
• Custom challenge solvers
• Certificate bundling
• OCSP helper function

Please keep in mind that CLI switches and APIs are still subject to change.

When using the standard --path option, all certificates and account configurations are saved to a folder .lego in the current working directory.