How HTTPS and TLS Really Work: The Handshake Explained

You added a certificate, the browser shows a padlock, and you move on. Then one day an API client fails with SSL_ERROR_RX_RECORD_TOO_LONG, a mobile app rejects your chain, or a redirect loop appears after enabling HSTS. Suddenly HTTPS feels like a black box.

This article opens that box. We will walk through what actually happens during a TLS handshake, why certificates exist, where symmetric and asymmetric encryption each fit, and how to debug the failures you will realistically hit in production.

What HTTPS Actually Is

HTTPS is not a separate protocol. It is plain HTTP sent over a TLS-encrypted connection. The URL scheme changes, the default port moves from 80 to 443, and everything else — methods, headers, status codes, cookies — behaves the same.

TLS gives you three guarantees on top of TCP:

• Confidentiality — intermediaries cannot read the traffic.
• Integrity — tampering in transit is detected.
• Authentication — the client can verify it is talking to the server that owns the domain.

That third guarantee is the one people underestimate. Encryption without authentication is worthless: an attacker who can intercept traffic could simply present their own key.

The TLS 1.3 Handshake, Step by Step

TLS 1.3 simplified the handshake dramatically compared to TLS 1.2. Here is the flow between a browser and a server:

1. ClientHello — the client sends supported TLS versions, cipher suites, a random value, and a key share for a Diffie-Hellman group (for example X25519).
2. ServerHello — the server picks a cipher suite and sends its own key share. Both sides can now derive the same shared secret.
3. Encrypted extensions and certificate — the server sends its certificate chain, already encrypted with the derived keys.
4. Finished — both sides send a MAC over the handshake transcript to prove nothing was tampered with.
5. Application data — HTTP requests and responses flow over the established session.

In TLS 1.3 the full handshake costs one round trip, and resumption can reach zero round trips. TLS 1.2 needed two round trips before any application data could flow.

Why the certificate is sent after keys are derived

In TLS 1.3 the certificate is encrypted. That means a passive observer cannot even see which domain the client is visiting from the certificate alone, which reduces information leakage. It also means debugging tools that expect a plaintext certificate in the first packets will look confused.

Symmetric vs Asymmetric Encryption

A common misconception is that HTTPS encrypts everything with the server's public key. It does not. Asymmetric cryptography is slow and size-limited; it is used only to establish a shared secret.

Diffie-Hellman key exchange has a valuable property called forward secrecy: even if the server's private key leaks later, recorded past sessions cannot be decrypted, because the session keys were never derived from that long-term key.

What Is Inside a Certificate

An X.509 certificate binds a public key to a domain name. It contains:

• The subject (domain) and subject alternative names (SANs), which is what browsers actually check.
• The public key and its algorithm.
• The validity period.
• The issuer and a signature from that issuer.

Browsers trust a fixed set of root certificate authorities. Your server sends an intermediate certificate, and the browser builds a chain from your leaf certificate up to a trusted root. If the intermediate is missing, desktop browsers may still work because they cache intermediates, while mobile clients and command-line tools fail. This is the single most common production TLS mistake.

Practical Debugging Steps

When HTTPS misbehaves, work from the outside in.

1. Check the chain with openssl s_client -connect example.com:443 -servername example.com -showcerts. Confirm the server sends the full chain, not just the leaf.
2. Check the expiry — expired certificates produce hard failures with no workaround.
3. Check the hostname — the SAN list must contain the exact hostname the client requests, including any www prefix.
4. Check the protocol version — if a legacy client only supports TLS 1.0 and your server requires TLS 1.2+, the handshake fails before HTTP.
5. Check redirects — a redirect from HTTPS to HTTP and back creates a loop; fix it at the server config, not in application code.

# Inspect the certificate chain and negotiated protocol
openssl s_client -connect example.com:443 \
  -servername example.com -showcerts < /dev/null 2>&1 | \
  openssl x509 -noout -subject -issuer -dates

# Test a specific TLS version
openssl s_client -connect example.com:443 -tls1_2 < /dev/null

In Nginx, the ssl_certificate directive should point to a file containing the leaf certificate followed by the intermediate, in that order. Concatenating them in the wrong order produces chain errors that are easy to miss.

HSTS and the Cost of Getting It Wrong

HTTP Strict Transport Security tells browsers to use HTTPS for a domain for a given period. It is powerful and unforgiving: once a browser has seen a long max-age, it will refuse plain HTTP even if you later disable HTTPS on the server.

Start with a short max-age, verify everything works, then increase it. Add includeSubDomains only after every subdomain serves valid HTTPS. Preloading is effectively permanent — only request it when you are certain.

Common Misconfigurations

• Mixed content — an HTTPS page loading scripts or images over HTTP. Browsers block active mixed content and warn on passive content.
• Weak cipher suites — leaving CBC-mode ciphers and old protocol versions enabled for compatibility you no longer need.
• Missing OCSP or CRL handling — revocation checking is unreliable in practice; short-lived certificates are the more robust answer.
• Certificate automation gaps — renewal works, but the reload step that makes the server pick up the new certificate is missing.

Treat certificate renewal as a deployment pipeline, not a calendar reminder. If renewal requires a human, it will eventually fail at the worst time.

FAQ

Does HTTPS encrypt the domain name I visit?

The domain appears in the TLS SNI extension and is usually visible on the network. The path and query string are encrypted, but a network observer can typically see which host you connected to.

Why does my site work in a browser but fail in curl or a mobile app?

Almost always a missing intermediate certificate. Browsers often cache intermediates from previous visits, while curl and many mobile HTTP stacks require the server to send the complete chain.

Is TLS 1.3 always better than TLS 1.2?

For modern clients, yes — fewer round trips, a smaller cipher suite set, and forward secrecy by default. Keep TLS 1.2 enabled only if you still serve clients that cannot negotiate 1.3.

When you need to inspect logs from the servers terminating TLS, a quick pass through the Nginx log analyzer helps you spot handshake failures, redirect loops, and unusual client behavior before they become outages.