HTTP, HTTPS & Protocols

What Happens After We Know the Server’s Address?

When you type a URL into your browser and press Enter, your computer has already done something remarkable: it translated that domain name into an IP address using DNS (remember from the previous section?). Now comes the next critical question: how does your computer actually communicate with that server?

The answer is HTTP—HyperText Transfer Protocol. It’s the language that web browsers and servers speak to each other. Every time you load a webpage, submit a form, or fetch data from an API, you’re using HTTP. It’s so fundamental to the web that understanding it is non-negotiable for system design. In this chapter, we’ll explore how HTTP works, why we need HTTPS, and how the protocol has evolved to handle modern internet demands.

Think of HTTP as the postal service of the internet: it defines a standardized format for how messages are packaged, sent, and received between clients (like your browser) and servers.

The Request-Response Dance

HTTP operates on a simple but elegant model: request-response. Your browser sends a request to the server, and the server sends back a response. Every interaction follows this pattern.

An HTTP request contains several components:

• Method: An action you want to perform (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS)
• URL/Path: The specific resource you’re requesting
• Headers: Metadata about the request (like “what language do I prefer?” or “who am I?”)
• Body: Optional data you’re sending (for POST or PUT requests)
• Version: Which version of HTTP you’re using (1.0, 1.1, 2.0, or 3.0)

The server responds with:

• Status Code: A number indicating success or failure (200, 404, 500, etc.)
• Headers: Metadata about the response (content type, caching instructions, cookies)
• Body: The actual content (HTML, JSON, images, etc.)

Let’s talk about HTTP methods—these are the verbs of the web:

Method	Purpose	Has Body?	Idempotent?
GET	Retrieve data	No	Yes
POST	Create new resource	Yes	No
PUT	Replace entire resource	Yes	Yes
DELETE	Remove resource	No	Yes
PATCH	Partially update resource	Yes	No
HEAD	Like GET, but no body	No	Yes

Status codes come in five families, and the first digit tells you everything:

• 2xx (Success): 200 OK, 201 Created, 204 No Content
• 3xx (Redirection): 301 Moved Permanently, 302 Found, 304 Not Modified
• 4xx (Client Error): 400 Bad Request, 401 Unauthorized, 404 Not Found
• 5xx (Server Error): 500 Internal Server Error, 503 Service Unavailable

Headers and cookies are how HTTP maintains state. HTTP itself is stateless—each request is independent. But we often need to remember who you are (authentication) or remember your preferences (personalization). That’s where cookies come in: small pieces of data stored on your browser that get sent with every request to a specific domain. Sessions layer on top of this, storing your login information on the server and referencing it via a session cookie.

Now, here’s the critical evolution: HTTPS. Regular HTTP sends everything in plain text. Anyone sniffing your network traffic could see passwords, credit card numbers, everything. HTTPS adds the “S” for Secure by encrypting the connection using TLS (Transport Layer Security, formerly called SSL). Before any HTTP communication happens, the client and server perform a TLS handshake: they negotiate encryption keys, verify each other’s identity via certificates, and establish a secure tunnel. Once that tunnel exists, all HTTP traffic flows through it encrypted.

A Restaurant Analogy for HTTP

Imagine you’re ordering at a restaurant. You (the client) are seated at a table, and you want to order food. You write down your order (the HTTP request) and hand it to the waiter (the HTTP protocol). The waiter carries your written order to the kitchen (the server), where the chef reads your request, prepares the food, and gives it back to the waiter with a note about whether it’s ready (the status code). The waiter brings your food to your table (the HTTP response with a body), along with details like a receipt and any special notes (headers).

What if the kitchen is busy and says “I’ll call you when it’s ready” (a 202 Accepted status)? What if you ask for something they don’t make (a 404 Not Found)? What if the waiter hands you someone else’s order by mistake (a network error)? This is HTTP in action: a standardized conversation with expected rules and responses.

How HTTP Has Evolved

HTTP/1.0 was simple but inefficient. Every single request required a new connection. Your browser would ask the server “Can I connect?” (TCP handshake), send a request, wait for a response, and then close the connection. For a webpage with 50 images, this meant 50 separate connection setups. Wasteful.

HTTP/1.1 introduced persistent connections: keep the connection open and send multiple requests through the same pipe. But there was a catch—head-of-line blocking. If you sent requests A, B, and C, but the server processed them slowly in order, request C had to wait for A and B, even if it could be processed faster.

Here’s a diagram showing the inefficiency:

sequenceDiagram
    participant Client
    participant Server
    Client->>Server: GET /page.html
    Server->>Client: 200 OK + HTML
    Client->>Server: GET /image1.jpg
    Server->>Client: 200 OK + Image
    Client->>Server: GET /image2.jpg
    Server->>Client: 200 OK + Image
    Note over Client,Server: Three sequential requests

HTTP/2 solved this with multiplexing: send multiple requests and responses interleaved on the same connection without blocking. Instead of waiting for image1 to finish before requesting image2, you send both requests immediately and get responses as they’re ready. HTTP/2 also introduced header compression (using HPACK) to reduce overhead, and server push, where the server can proactively send resources it predicts you’ll need.

Compare the efficiency:

sequenceDiagram
    participant Client
    participant Server
    Client->>Server: GET /page.html
    Client->>Server: GET /image1.jpg
    Client->>Server: GET /image2.jpg
    Server->>Client: 200 OK + HTML
    Server->>Client: 200 OK + Image1
    Server->>Client: 200 OK + Image2
    Note over Client,Server: Multiplexed—all at once

HTTP/3 represents a fundamental shift. It’s based on QUIC, which runs over UDP instead of TCP. Why? TCP is old and slow. Setting up a TCP connection requires a handshake (3 steps), then a TLS handshake (additional steps). QUIC merges these: the initial connection and TLS handshake happen simultaneously in one round trip. Plus, QUIC is more resilient—if you switch from WiFi to cellular mid-request, the connection survives because it’s identified by a connection ID, not by the IP address. This is huge for mobile users.

The TLS handshake deserves its own spotlight. Here’s what happens:

1. Client Hello: Your browser sends supported cipher suites, protocol versions, and a random number
2. Server Hello: The server picks a cipher suite, sends its certificate (proving it is who it claims to be), and its own random number
3. Key Exchange: Client and server compute a shared secret using the random numbers (using algorithms like Diffie-Hellman)
4. Finished: Both sides send a message proving they can encrypt and decrypt with the shared secret

Once complete, everything is encrypted.

For designing APIs and services, REST (Representational State Transfer) has become the standard. REST principles lean on HTTP’s strengths: use the right HTTP method (POST for creation, PUT for updates, DELETE for deletion), return appropriate status codes, and structure URLs as nouns, not verbs. Instead of /user/create, use POST /users. Instead of /user/update/123, use PUT /users/123.

Real API Design in Action

Let’s say you’re building a social media API. Here’s what a proper HTTP conversation might look like:

Request:
POST /api/v1/posts HTTP/1.1
Host: api.socialapp.com
Content-Type: application/json
Authorization: Bearer eyJhbGc...

{
  "content": "Hello, world!",
  "visibility": "public"
}

Response:
HTTP/1.1 201 Created
Content-Type: application/json
Location: /api/v1/posts/789

{
  "id": 789,
  "content": "Hello, world!",
  "visibility": "public",
  "created_at": "2024-02-10T14:30:00Z",
  "author_id": 456
}

Notice the 201 status (Created, not just 200). Notice the Location header pointing to the newly created resource. Notice the Authorization header with a bearer token for authentication.

When should you use HTTP/2 versus HTTP/3? HTTP/2 has excellent browser support (99%+ of traffic now). HTTP/3 is newer, with about 30% adoption and growing. Use HTTP/3 if you’re building mobile-first services where connection switching matters. Use HTTP/2 if you need maximum compatibility today. Most production systems support both—the client and server negotiate the best version they both understand.

Performance differences are real but context-dependent. HTTP/2 can reduce load times 15-30% compared to HTTP/1.1 by eliminating head-of-line blocking. HTTP/3 adds another 10-20% improvement, especially on high-latency or lossy networks (3G, satellite, etc.). But optimization isn’t just about the protocol—database queries, caching, and code efficiency matter far more.

Trade-offs and Strategic Decisions

Here’s the tension: HTTP/2 is battle-tested and everywhere. Switching to HTTP/3 requires infrastructure support, and not all CDNs or hosting providers support it equally. Some proxies and firewalls don’t understand QUIC yet. For a greenfield project, HTTP/3 is worth investing in; for maintaining a large existing system, the migration cost might outweigh benefits.

REST has dominated for 15 years, but GraphQL and gRPC are challenging it. REST gives you a fixed set of fields per endpoint—if you need just a user’s name, you still get their entire profile. GraphQL lets you request exactly what you need, reducing bandwidth for mobile clients. gRPC is optimized for service-to-service communication, much faster than REST. The trade-off? Complexity. REST is simple; GraphQL and gRPC require more infrastructure.

Persistent HTTP connections solve the efficiency problem but create new ones: server resources. Each open connection consumes memory. A server with 100,000 concurrent connections needs careful resource management. This is why connection pooling exists—your client library maintains a pool of connections to reuse rather than creating new ones constantly.

Key Takeaways

• HTTP is a request-response protocol; every interaction follows this pattern with methods (GET, POST, PUT, DELETE), status codes, headers, and optional bodies
• HTTPS wraps HTTP in TLS encryption, protecting data in transit via a handshake that establishes shared encryption keys
• HTTP/1.1 introduced persistent connections but suffered from head-of-line blocking; HTTP/2 solved this with multiplexing and header compression
• HTTP/3 uses QUIC (UDP-based) for faster handshakes, connection migration, and better resilience on unstable networks
• REST API design uses HTTP methods and status codes correctly, treating URLs as nouns and leaning on standard HTTP semantics
• Choose your protocol version and architectural style based on your audience (mobile vs desktop), infrastructure maturity, and performance requirements

Practice Scenarios

Scenario 1: Diagnosing a Slow API Your team reports that an API endpoint is slow. You discover clients are using HTTP/1.1 with 50 sequential requests. How would HTTP/2 or HTTP/3 change this? What other factors might you investigate (caching, database queries, payload size)?

Scenario 2: Building an Authentication Flow Design an HTTP-based authentication system using cookies and sessions. Sketch the request/response flow for login, a subsequent authenticated request, and logout. What headers and status codes are important?

Scenario 3: Protocol Selection You’re building a real-time collaboration tool (like Google Docs). Should you use HTTP for everything, or would WebSockets (covered in the next section) be better? What are the trade-offs?

What’s Next?

We’ve explored how HTTP carries data across the internet, from the initial connection to the encrypted exchange of requests and responses. But HTTP is request-response—what happens when you need true bidirectional, real-time communication? That’s where WebSockets for Real-Time Communication comes in, letting server and client push messages to each other instantly without the overhead of repeated HTTP handshakes.