DigitalOcean load balancers, and what's behind them
A DigitalOcean load balancer distributes traffic to a group of backend resources, so the health of a service does not depend on a single server. Duskwatch is an independent iPhone app for DigitalOcean, in development; it shows each load balancer's state, which droplets behind it are off, its forwarding rules, health check and settings, and flags what would turn visitors away.
Updated
The droplets behind it
A load balancer is only as healthy as what it sends traffic to. The load balancer screen lists the droplets behind it, whether they are assigned by name or by tag, each with its own state. When some are off, a finding says how many, and the others carry the traffic in the meantime.
1 of 2 droplets behind it is off · web-02 is off · from the droplet status, not the health check
No droplets behind it · No droplet has the tag web
One honest limit: the load balancer's API object reports the health check's settings but not its result for each droplet. So the state you see is each droplet's own status, and the finding says so in its evidence. A droplet that is running but failing the health check, say because its web server stopped, looks fine here; an uptime check on the site catches that case.
Forwarding rules, health check and settings
Below the droplets sit the forwarding rules, from the entry protocol and port to the target, and a calm empty state when there are none: without a rule a load balancer accepts no traffic. The health check section shows what is checked, how often, and after how many failures or passes a droplet is taken out or put back. The settings cover the type (regional, network or global), size, redirect from HTTP to HTTPS, sticky sessions, PROXY protocol and backend keepalive.
| Shown as | Meaning | Example |
|---|---|---|
| Checks | The protocol, port and path it requests | HTTP, port 80, path / |
| Every | Seconds between health checks | 10 seconds |
| Unhealthy after | Consecutive failures before traffic stops | 3 checks |
| Healthy again after | Consecutive passes before traffic returns | 5 checks |
The unhealthy threshold is how many consecutive times a node must fail a health check before the load balancer stops forwarding traffic to it; the healthy threshold is how many passes it needs before traffic returns.
DigitalOcean docs: manage regional load balancers (external link), verified
An HTTP or HTTPS health check succeeds on a status code from 200 to 399; a TCP health check succeeds when the TCP handshake completes.
DigitalOcean docs: manage regional load balancers (external link), verified
Certificates on HTTPS rules
An HTTPS rule terminates TLS with a certificate stored in your team. That certificate has its own screen, which lists lb-web under "Used by", so a certificate that ends in 9 days is traced to the load balancer it would break. Let's Encrypt certificates renew on their own while DigitalOcean manages the domain's DNS; custom ones need a new upload. More on the certificates page.
With DigitalOcean DNS managing the domain, a load balancer can use a Let's Encrypt certificate that DigitalOcean creates and renews automatically.
DigitalOcean docs: configure SSL termination (external link), verified
Common causes of unhealthy backends
When visitors get errors but every droplet is on, the cause is usually inside the droplet or between it and the load balancer. DigitalOcean's troubleshooting guide lists the usual suspects, and the droplet screen helps you check the first two.
Common causes of failed health checks include backend software that is frozen or not responding, a firewall on the droplet blocking the port, a service not listening on the private network interface, an HTTP check getting an error status, and a PROXY protocol mismatch.
DigitalOcean docs: troubleshoot load balancer health checks (external link), verified
- Open each droplet behind it for its metrics, recent restarts and firewall findings.
- Hand off to your SSH app to check the service itself.
- Open the load balancer in DigitalOcean to change rules or the health check.
What Duskwatch doesn't do here
- No health check result per droplet, because the load balancer's API object does not report it.
- No push of its own: a droplet behind it that goes off is covered by the droplet-off push.
- No editing forwarding rules, health checks or settings.
- No creating, resizing or deleting load balancers.
Questions
How do I know a backend droplet is down?
DigitalOcean stops sending traffic to a droplet after it fails the health check a set number of times in a row, the unhealthy threshold. Duskwatch shows the droplets behind the load balancer that are off, from each droplet's own status.
Will I get a push when a load balancer has a problem?
Not for the load balancer itself; its problems are findings in the app. A droplet behind it that goes off sends the droplet-off push, if you turn alerts on.
Which load balancer settings can I see?
Forwarding rules, the health check, type, size, HTTP to HTTPS redirect, sticky sessions, PROXY protocol and backend keepalive. You change them in DigitalOcean.
Why does it show droplet status instead of the health check?
The load balancer's API object reports the health check's settings, not its result for each droplet. The finding says that its evidence comes from the droplet status.