Load Balancing Multiple VPS Servers with HAProxy

One VPS handles more traffic than most sites will ever see — until it doesn't. When you've scaled the server itself as far as it'll go and the site still chokes under load, or when a single point of failure is no longer acceptable, the next step isn't a bigger box. It's more than one box, with something in front deciding where each request goes. That's HAProxy's job.

When One VPS Isn't Enough

Two situations push people toward load balancing. The first is pure traffic: CPU and RAM are maxed on the current VPS, you've already tuned PHP-FPM and MySQL, and the only way forward is running the app on two or three servers at once. The second is uptime: if that one VPS reboots, gets patched, or has a hardware fault, the site goes down with it, and that's not acceptable for something processing orders or handling client logins.

A load balancer solves both. It sits in front of two or more backend servers, spreads requests across them, and — critically — stops sending traffic to any backend that stops responding. Add or remove a backend server without anyone noticing.

What HAProxy Actually Does

HAProxy is a dedicated load balancer and proxy — not a general web server like Nginx or Apache. It listens on your public IP (usually ports 80/443), and for every request it picks one of your backend app servers based on a chosen algorithm, forwards the request there, and streams the response back. It also runs health checks against each backend on its own schedule, independent of the requests it's routing, so it knows within seconds if a server has gone down.

You can run HAProxy on its own small VPS dedicated to just this job, or on one of your existing app servers if traffic is light enough. For anything production-facing, a separate box is worth the extra few dollars a month — if the load balancer VPS itself falls over, everything behind it becomes unreachable too.

Installing HAProxy

On Ubuntu/Debian:

sudo apt update
sudo apt install haproxy -y
haproxy -v

On AlmaLinux/CentOS:

sudo dnf install haproxy -y
sudo systemctl enable --now haproxy

The config file lives at /etc/haproxy/haproxy.cfg. Back up the default before touching it:

sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.orig

Basic Config: Two Backend Servers

Here's a minimal working setup balancing traffic between two app servers on your private network:

frontend web_front
    bind *:80
    default_backend web_back

backend web_back
    balance roundrobin
    option httpchk GET /health
    server web1 10.0.0.11:80 check
    server web2 10.0.0.12:80 check

Test the config syntax before reloading — HAProxy will refuse to start on a bad config, but you want to know that before you're staring at a downed site:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxy

Point your domain's A record at the HAProxy server's public IP, not at either backend directly. The backends should only be reachable from the load balancer — lock that down with your firewall (UFW or CSF) so nobody can bypass HAProxy and hit a backend server straight.

Health Checks: Don't Send Traffic to a Dead Server

The option httpchk GET /health line tells HAProxy to request that path on each backend periodically. If a backend stops responding — app crashed, PHP-FPM died, server rebooting for patches — HAProxy marks it down and stops routing traffic there within a few seconds, automatically, no manual intervention.

Two things trip people up here:

• No /health endpoint exists. Point it at something lightweight instead — a static file, or a simple route that just returns HTTP 200 without touching the database. Don't health-check your homepage; if the database is slow, you don't want that dragging down the health check too.
• Health checks pass but the app is still broken. A check that only confirms the web server answers won't catch "PHP is up but MySQL connections are exhausted." For anything beyond a static site, build a real health endpoint that checks the dependencies you actually care about.

Watch checks fail and recover live in the logs:

sudo tail -f /var/log/haproxy.log

Choosing a Load-Balancing Algorithm

The balance directive controls how requests get distributed. The three you'll actually use:

For most WordPress or general web app setups behind a load balancer, leastconn tends to behave better than roundrobin once traffic gets uneven — a slow admin-ajax request on one server shouldn't keep piling more work onto it just because it's "next in line."

Sticky Sessions for Logins and Carts

If your app stores session data (login state, shopping cart) in local files or local memory on each server instead of a shared store like Redis or a database, a user bouncing between backend servers on different requests will get logged out or lose their cart randomly. That's the single most common complaint after a first load-balancer deploy.

Two ways to fix it, and one is clearly better long-term:

• Sticky sessions (cookie-based): HAProxy pins a visitor to one backend for the duration of their session. Quick fix, add this to the backend block:

  backend web_back
      balance roundrobin
      cookie SERVERID insert indirect nocache
      server web1 10.0.0.11:80 check cookie web1
      server web2 10.0.0.12:80 check cookie web2

• Shared session storage: move sessions to Redis or the database so any backend can serve any request. This is the real fix — it's what lets you actually take a server out of rotation for maintenance without logging anyone out. Sticky sessions just paper over the problem.

SSL Termination at the Load Balancer

Terminate SSL at HAProxy rather than on each backend — one certificate to renew, not three. Combine the cert and key into a single PEM file (Let's Encrypt via Certbot works fine here):

sudo cat /etc/letsencrypt/live/example.com/fullchain.pem \
  /etc/letsencrypt/live/example.com/privkey.pem > /etc/haproxy/certs/example.com.pem

frontend web_front
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem
    redirect scheme https if !{ ssl_fc }
    default_backend web_back

Traffic from HAProxy to the backends can then travel as plain HTTP over your private network — that's fine as long as that network isn't public. If the backends are on separate physical hosts reachable over the internet rather than a private VPC, re-encrypt that hop too.

Watching It Work: The HAProxy Stats Page

Enable the built-in stats dashboard to see, in real time, which backends are up, how many connections each is handling, and whether health checks are passing:

listen stats
    bind *:8404
    stats enable
    stats uri /haproxy?stats
    stats auth admin:a-real-password-here

Restrict access to this port at the firewall level — it's not meant to be public. It's the fastest way to confirm a config change did what you expected before you find out the hard way from a support ticket.

Common Problems

Symptom: HAProxy starts, but every request returns a 503 with "no server available."
Cause: Every backend is failing its health check, usually because the health-check path doesn't exist or a firewall rule between HAProxy and the backends is blocking the request.
Fix: curl the health-check URL directly from the HAProxy server (curl http://10.0.0.11/health) to confirm it's reachable and returns 200. Check UFW/security group rules between the two servers next.

Symptom: Traffic all lands on one backend even though both show as healthy.
Cause: source balancing with a small pool of client IPs (common behind a corporate NAT or another proxy in front of HAProxy) can hash almost everyone to the same server.
Fix: Switch to roundrobin or leastconn unless you specifically need IP-based stickiness.

Symptom: Site works, but users get randomly logged out.
Cause: No shared session storage and no sticky sessions configured, so consecutive requests land on different backends with different local session data.
Fix: Add cookie-based stickiness as a stopgap, then move sessions to Redis or the database.

Prevention Checklist

• Lock down backend servers so they only accept connections from the HAProxy server's IP.
• Build a real /health endpoint that checks the dependencies your app actually needs, not just "the web server responded."
• Move sessions to shared storage before you scale past one backend — don't rely on sticky sessions as a permanent fix.
• Monitor the HAProxy stats page or wire it into your existing monitoring so a backend going down triggers an alert, not silence.
• Test failover deliberately: stop the app on one backend and confirm HAProxy routes around it within your expected timeframe.