Evolution of Software Systems

From One Block to Many Pieces

Imagine you’re building a pizza restaurant. On day one, you do everything yourself—you take orders, prepare dough, cook pizzas, and deliver them. That works fine when you’re receiving 10 orders a day. But what happens when you grow to 100 orders daily? You become the bottleneck. You can’t scale by hiring more “you.” You need to divide responsibilities: someone takes orders, another person makes dough, someone else operates the oven, and another handles deliveries. This is exactly how software systems have evolved over the past two decades.

In this chapter, we’ll explore how software architectures have transformed from monolithic applications—where everything is bundled together—to distributed systems that break functionality into independent, scalable components. Understanding this evolution isn’t just historical trivia; it directly impacts the design decisions you’ll make as a system designer. By the end of this chapter, you’ll understand the journey from monoliths to microservices, the reasons behind each evolution, and when to use each approach.

This evolution is central to system design because it teaches us a fundamental principle: how you structure your systems determines your ability to scale, maintain, and evolve them. As you learned in Section 1, system design is about building systems that work reliably at scale. The architectural patterns we’ll discuss here form the foundation for achieving that goal.

Monoliths, SOA, and Microservices

What is a Monolithic Architecture?

A monolithic architecture is a single, tightly integrated application where all business logic, user interfaces, and data access layers are bundled into one codebase and deployed as a single unit. Think of it as one massive block of stone—hence “monolith.” When you update a small feature, you rebuild and redeploy the entire application. All components share the same database, programming language, and runtime environment.

In the early days of web development, monoliths made sense. They were simpler to develop, test, and deploy. A single developer could understand the entire system. Database transactions were straightforward because everything ran in one process. Netflix, Amazon, and Twitter all started with monolithic architectures. The monolith wasn’t a failure—it was the right choice for their time.

Why Did We Need Something More?

As applications grew, monoliths hit a critical scaling problem. If your shopping cart service handles 10,000 requests per second, but your recommendation engine only needs 1,000 requests per second, you still have to scale your entire application together. You’re buying more hardware for components that don’t need it. More critically, if the recommendation engine has a bug that causes it to crash, it takes down your entire application—including the shopping cart that was working perfectly.

Monoliths also became development bottlenecks. Teams working on different features had to coordinate and merge changes in the same codebase constantly. Deploying one small change meant testing the entire system. If something broke, it took longer to identify which team’s changes caused the problem.

Enter: Service-Oriented Architecture (SOA)

To address these problems, architects introduced Service-Oriented Architecture (SOA). SOA breaks a monolith into loosely coupled services that communicate over a network using standardized protocols. Instead of one pizza restaurant, imagine a network of specialized shops: one that makes dough, one that handles sauce, one that manages cheese, and one that does final assembly. Each shop can scale independently, be maintained by different teams, and even use different techniques.

In SOA, services are typically large and business-function oriented. You might have an “Order Service,” “Inventory Service,” “Billing Service,” and “Shipping Service.” These services communicate through enterprise messaging systems (like Apache Kafka or RabbitMQ) or web services (like SOAP). This approach provided more flexibility than monoliths but introduced new complexity: network latency, distributed transactions, and the challenge of managing many services simultaneously.

The Microservices Revolution

Microservices took the lessons from SOA and pushed them further. Instead of large business-domain services, microservices are tiny, focused services that do one thing exceptionally well. Following the Unix philosophy of “do one thing and do it well,” a microservices architecture might have separate services for: user authentication, product catalog, shopping cart, payment processing, order management, inventory, and notifications.

Each microservice is independently deployable, scalable, and maintainable. They communicate through lightweight protocols like HTTP/REST or gRPC. A team of 5-10 people (what Amazon calls “two-pizza teams”) typically owns one microservice. This ownership model is crucial—teams can innovate faster because they’re not blocked by other teams’ deployment schedules.

Modern Distributed Systems

Today, we use the term “distributed systems” as an umbrella for any architecture where components run on multiple machines and communicate over a network. This includes microservices, but also serverless architectures (AWS Lambda, Google Cloud Functions), event-driven systems, and hybrid approaches. The key characteristics are: components are separated by network boundaries, they can fail independently, and they must coordinate to achieve business goals.

The Post Office Parallel

Think of how mail delivery has evolved. In the past (monolith era), the post office handled everything: accepting letters, sorting them, loading trucks, and delivering them. One problem anywhere meant the entire system failed. Today’s mail system is more distributed. Some companies specialize in letter collection (mailboxes), others in regional sorting, and still others in last-mile delivery. Each part can operate independently, scale based on demand, and even use different technologies. If one regional hub is overloaded, other hubs continue working. This parallel to how modern software systems operate: independent, specialized components working together toward a shared goal.

Under the Hood

Monolithic Architecture in Detail

In a monolith, when a user places an order, the request flows through a single application. Here’s what happens:

User Request → Load Balancer → Application Instance → Business Logic Layer → Data Access Layer → Database

All the logic—validating user input, checking inventory, calculating taxes, processing payment, and updating the database—lives in one codebase. The application instance is stateless (no stored data), but all instances connect to the same database.

graph TB
    Client["Client Browser"]
    LB["Load Balancer"]
    App1["Application Instance 1"]
    App2["Application Instance 2"]
    App3["Application Instance 3"]
    DB["Single Database"]

    Client -->|HTTP Request| LB
    LB -->|Route| App1
    LB -->|Route| App2
    LB -->|Route| App3
    App1 -->|SQL Queries| DB
    App2 -->|SQL Queries| DB
    App3 -->|SQL Queries| DB

Advantages: Simple deployment pipeline, ACID transactions are straightforward, debugging is easier because components are tightly coupled. Disadvantages: Scaling is inefficient, a bug in one module can crash everything, technology choices are locked (can’t use Python for one part and Java for another).

Service-Oriented Architecture (SOA) in Detail

SOA introduces the concept of service boundaries. Each service owns its business domain:

graph TB
    Client["Client"]
    ESB["Enterprise Service Bus<br/>or API Gateway"]
    OrderSvc["Order Service"]
    InventorySvc["Inventory Service"]
    BillingSvc["Billing Service"]
    ShippingSvc["Shipping Service"]
    OrderDB["Order DB"]
    InventoryDB["Inventory DB"]
    BillingDB["Billing DB"]

    Client --> ESB
    ESB --> OrderSvc
    ESB --> InventorySvc
    ESB --> BillingSvc
    ESB --> ShippingSvc
    OrderSvc --> OrderDB
    InventorySvc --> InventoryDB
    BillingSvc --> BillingDB
    OrderSvc -.->|SOAP/Async Message| InventorySvc
    OrderSvc -.->|Async Message| BillingSvc

Each service has its own database and communicates with other services through an Enterprise Service Bus (ESB) or messaging system. This decoupling allows independent scaling and deployment. However, distributed transactions become complex—if the payment service fails after the order is created, you need logic to handle that failure gracefully.

Microservices Architecture in Detail

Microservices push SOA’s decoupling further:

graph TB
    Client["Client Application"]
    Gateway["API Gateway"]
    Auth["Auth Service"]
    Product["Product Service"]
    Cart["Cart Service"]
    Order["Order Service"]
    Payment["Payment Service"]
    Notify["Notification Service"]

    AuthDB["Auth DB"]
    ProductDB["Product DB"]
    CartDB["Cart DB"]
    OrderDB["Order DB"]

    Client --> Gateway
    Gateway --> Auth
    Gateway --> Product
    Gateway --> Cart
    Gateway --> Order
    Auth --> AuthDB
    Product --> ProductDB
    Cart --> CartDB
    Order --> OrderDB
    Order -->|REST/gRPC| Payment
    Order -->|Event: Order.Created| Notify

Each service is independently deployable. Services communicate via REST APIs, gRPC, or event streams. The beauty here: if the Notification Service is down, users can still place orders. The order system posts an event (“Order Created”), and whenever the Notification Service comes back online, it processes pending events.

Deployment and Operation

Monolithic deployment is straightforward: build once, deploy everywhere. Microservices deployment requires orchestration platforms like Kubernetes. Each service might have different resource requirements, scaling policies, and deployment frequencies:

Aspect	Monolith	SOA	Microservices
Deployment Unit	Entire application	Large service	Single service
Deployment Frequency	Weekly/Monthly	Weekly	Multiple times daily
Database	Shared	Per-domain	Per-service
Inter-service Communication	In-process	ESB/Messaging	REST/gRPC/Events
Scaling Granularity	Entire app	Large service	Individual service
Team Size	Larger, shared ownership	Medium	Small (2-pizza teams)

Lessons from Twitter, Amazon, and Netflix

Twitter’s Evolution: A Case Study

Twitter started as a Rails monolith in 2006. When the platform exploded in popularity, the monolith became a bottleneck. Twitter evolved to a service-oriented architecture with specialized services: the tweet-writing service, the timeline service, the search service, and the notification service. Each service has different scaling needs—the timeline service handles far more reads than writes, while the search service requires different indexing strategies.

Here’s a simplified view of how Twitter’s order service might work today in a microservices model:

# Order Service Configuration (Simplified)
apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  replicas: 5
  containers:
  - name: order-service
    image: twitter/order-service:v2.1
    resources:
      cpu: 500m
      memory: 512Mi
    env:
    - name: PAYMENT_SERVICE_URL
      value: "http://payment-service:8080"
    - name: INVENTORY_SERVICE_URL
      value: "http://inventory-service:8080"
  ports:
  - containerPort: 8080

Amazon’s Two-Pizza Team Principle

Jeff Bezos famously mandated that Amazon teams should be sized such that they can be fed with two pizzas. This forced the company toward microservices architecture: each team needed to own independent services that they could build, deploy, and operate without constant coordination with other teams. Amazon’s architecture became the blueprint for microservices—and their scale (handling millions of orders daily) proved the model could work at massive scale.

Netflix: Chaos Engineering and Independent Services

Netflix pioneered the practice of “chaos engineering”—intentionally breaking services to verify the system continues functioning. This only became practical with microservices. They developed tools like Hystrix (fault tolerance) and Chaos Monkey (random failure injection) because with hundreds of microservices, failures aren’t a matter of if but when. Their approach: design assuming services will fail, and verify your system degrades gracefully.

Choosing the Right Architecture

Monolith Strengths and Weaknesses

Monoliths excel for small teams (< 10 people) building new products with uncertain requirements. They’re faster to build initially, easier to reason about, and deployment is simpler. Use monoliths when: your team is small, your product is new, you’re experimenting with the business model, or your system genuinely doesn’t need to scale independently.

Pro tip: Many successful companies stayed with monoliths far longer than you’d expect. Basecamp (formerly 37signals) has publicly argued that monoliths are often over-engineered solutions for most business applications. Don’t prematurely optimize for scale you don’t have.

However, monoliths become problematic around 20-50 engineers or millions of users where different components need different scaling strategies. You hit walls: deployment bottlenecks, performance issues, inability to adopt new technologies, and high blast radius (one bug affects everything).

Microservices Strengths and Weaknesses

Microservices shine when you have: multiple teams working independently, extreme scaling requirements for specific services, technology diversity needs, or rapid deployment requirements. Companies like Netflix and Amazon deploy thousands of times daily—only feasible with microservices.

The trade-off is significant operational complexity. You need: comprehensive logging and tracing, sophisticated service discovery, circuit breakers and timeouts, comprehensive monitoring, and a strong deployment pipeline. You’re trading development simplicity for operational flexibility. A distributed system has failure modes that don’t exist in monoliths: network partitions, eventual consistency, cascading failures.

The Common Mistake: Premature Microservices

The biggest pitfall we see: teams adopting microservices before they’ve outgrown monoliths. They create dozens of tiny services without proper infrastructure for logging, monitoring, or service discovery. The result: a “distributed monolith”—all the complexity of microservices without the benefits. Debugging becomes a nightmare because requests flow through ten services, and you don’t have proper tracing.

Did you know? Many organizations that migrated to microservices later consolidated back to monoliths or hybrid approaches. The lesson: architecture should follow your growth, not precede it. Start with a monolith, build great instrumentation, and migrate specific services to separate deployments only when you have a concrete scaling or team-structure reason.

Key Takeaways

• Monoliths are not failures—they’re the appropriate starting point for most new applications. They become problematic at scale (20+ engineers, millions of users, diverse scaling needs).

• Service-Oriented Architecture introduced service boundaries, allowing independent scaling and deployment, but introduced operational complexity around distributed transactions and service coordination.

• Microservices are SOA’s evolution, emphasizing smaller, single-purpose services and enabling rapid, independent deployment. They require substantial operational maturity (logging, monitoring, tracing, service discovery).

• Each architecture makes trade-offs: monoliths trade scaling flexibility for simplicity; microservices trade development simplicity for operational flexibility and scaling capability.

• The migration path is real: successful companies evolved from monoliths → SOA → microservices as their needs demanded, not preemptively. Your architecture should match your current constraints, not hypothetical future needs.

• Operational capabilities determine viability: microservices are only feasible if you have comprehensive logging, monitoring, and deployment automation. Without these, you’ll build a distributed monolith that has all the downsides with none of the benefits.

Put It Into Practice

Scenario 1: The Growing Startup

You’re architecting the backend for a marketplace (similar to Uber or Airbnb) that currently has 50,000 users and is growing at 20% monthly. Your team has 15 engineers. The CTO asks: “Should we build a microservices architecture so we can scale?”

How would you advise? Consider: your team size, current scale, deployment frequency, and the complexity you’re willing to manage operationally. What would be the risks of jumping to microservices too early? What would be the risks of staying with a monolith if you’re expecting rapid growth?

Scenario 2: The Monolith Migration

Your company has a mature Node.js monolith that’s been running for 5 years. It serves 500 million requests daily. You’ve identified that the recommendation engine is becoming a bottleneck—it consumes 60% of CPU while serving only 15% of requests. However, you don’t have comprehensive logging or monitoring set up yet.

What’s your migration strategy? Would you immediately extract the recommendation engine into its own microservice, or would there be preparatory work? What operational challenges would you anticipate, and how would you address them?

Scenario 3: Technology Diversity

Your organization has three teams: Team A (Java specialists), Team B (Python specialists), and Team C (Go specialists). Team A is frustrated because the monolith is written in Java, and even simple services are built in Java using complex frameworks. They propose breaking into microservices so each team can use their preferred language.

Is this a good reason to migrate to microservices? What other factors should drive this decision? How might you address the teams’ concerns without requiring a full microservices migration?

What Comes Next

Now that you understand how software systems have evolved and why architectural decisions matter, we’re ready to explore the key system characteristics that drive design decisions. In the next chapter, “Key System Characteristics: Scale, Performance, and Reliability,” we’ll examine the specific requirements you’ll evaluate when designing systems: How many users? How much data? What latency requirements? These characteristics directly determine whether a monolith, SOA, or microservices approach is appropriate for your specific problem. The evolution you’ve learned here provides context; the characteristics we’re about to explore provide the decision framework.