Running Trojan and Nginx on the Same VPS with HAProxy

If you want to host a normal HTTPS website and run a Trojan server on the same VPS, you will quickly run into a classic problem:
only one process can bind to port 443 at a time.

This article documents a clean and production-ready solution using HAProxy with SNI-based TCP routing, allowing:

• Nginx to continue serving your website over HTTPS
• Trojan to run on the same port 443, without conflicts
• Let’s Encrypt certificates to renew automatically without downtime

The setup has been tested in real-world conditions and is suitable for long-term use.

Architecture Overview

The core idea is simple:

• HAProxy exclusively binds to :443
• Incoming TLS connections are routed based on SNI
• Traffic is forwarded internally to either Nginx or Trojan

Client
  |
  |  TLS (SNI)
  v
HAProxy :443
  |----------------------|
  |                      |
trojan.xxx.com       other domains
  |                      |
Trojan :24443        Nginx :8443

DNS

• xxx.com, www.xxx.com → VPS public IP
• trojan.xxx.com → VPS public IP

Ports

Firewall Rules (UFW)

A minimal and secure firewall configuration:

sudo ufw default deny incoming
sudo ufw default allow outgoing

sudo ufw allow 22022/tcp comment 'SSH'
sudo ufw allow 80/tcp comment 'HTTP (ACME)'
sudo ufw allow 443/tcp comment 'HTTPS'
sudo ufw enable

Nginx: ACME-Friendly HTTP Configuration

To avoid certificate renewal failures, do not redirect ACME requests.
Instead, serve them from a fixed webroot.

sudo mkdir -p /var/www/acme/.well-known/acme-challenge
sudo chown -R www-data:www-data /var/www/acme

trojan.xxx.com (ACME only)

server {
    listen 80;
    server_name trojan.xxx.com;

    location ^~ /.well-known/acme-challenge/ {
        root /var/www/acme;
        try_files $uri =404;
    }

    location / { return 404; }
}

Main website

server {
    listen 80;
    server_name xxx.com www.xxx.com;

    location ^~ /.well-known/acme-challenge/ {
        root /var/www/acme;
        try_files $uri =404;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

Reload Nginx:

sudo nginx -t && sudo systemctl reload nginx

Let’s Encrypt (Certbot)

Issue certificates using the webroot method:

sudo certbot certonly --webroot -w /var/www/acme \
  -d trojan.xxx.com \
  -m [email protected] --agree-tos --non-interactive

Automatic Reload After Renewal

Trojan supports reloading certificates via SIGUSR1:

sudo tee /etc/letsencrypt/renewal-hooks/deploy/reload-trojan.sh <<'EOF'
#!/bin/sh
systemctl kill -s SIGUSR1 trojan || systemctl restart trojan
EOF
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-trojan.sh

Trojan Configuration

Trojan must not bind to port 443 directly.

{
  "run_type": "server",
  "local_addr": "127.0.0.1",
  "local_port": 24443,
  "password": ["YOUR_PASSWORD"],
  "ssl": {
    "cert": "/etc/letsencrypt/live/trojan.xxx.com/fullchain.pem",
    "key": "/etc/letsencrypt/live/trojan.xxx.com/privkey.pem",
    "alpn": ["http/1.1"]
  }
}

Restart Trojan:

sudo systemctl restart trojan

HAProxy: The Core of the Setup

HAProxy performs pure TCP passthrough and routes connections by SNI.

frontend https
    bind *:443
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

    use_backend trojan if { req.ssl_sni -i trojan.xxx.com }
    default_backend nginx

backend trojan
    mode tcp
    server trojan 127.0.0.1:24443

backend nginx
    mode tcp
    server nginx 127.0.0.1:8443

Validate and restart:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl restart haproxy

Verification

Check certificate and SNI routing:

openssl s_client -connect trojan.xxx.com:443 -servername trojan.xxx.com </dev/null

If the certificate CN/SAN is correct, the routing works.

Common Pitfalls

• bind: Address already in use
  → Something other than HAProxy is still binding to 443.
• use_certificate_chain_file: No such file
  → Certificate paths in Trojan config are incorrect.
• TLS handshake failures
  → Wrong SNI, wrong domain, or certificate mismatch.

Client Example (Clash)

proxies:
  - name: trojan-443
    type: trojan
    server: trojan.xxx.com
    port: 443
    password: YOUR_PASSWORD
    sni: trojan.xxx.com

Final Notes

This architecture cleanly separates responsibilities:

• HAProxy: traffic dispatch
• Nginx: web hosting
• Trojan: proxy service

It avoids port conflicts, supports automatic certificate renewal, and scales well.

If you are running multiple TLS-based services on a single VPS, SNI-based TCP routing with HAProxy is the most robust solution.