The Future is Microservices in the Cloud

I talked a little bit yesterday about trying to cluster node.js so it will fill a VM and before that the relentless change from giant, monolithic application stacks running on enormous Java servlet containers to light SOA-based services.  I still believe node.js cluster is essential but...

Public clouds put new constraints on systems not felt since the early days of embedded programming.  Back in the days of yore when we were younger and better looking, memory was at a premium and systems always came 1U.  Computer architecture is cyclic; for a while single systems get bigger until they cannot grow vertically any more and then they fall over and duplicate then grow again.  We have two paradigms.

Enterprise Computing

  • Large, single systems
  • Little redundancy
  • Bare metal
  • High cores, high RAM
  • Multi-user tenancy 

Cloud Computing

  • Small and many systems
  • High redundancy
  • Virtualized
  • Few cores, little RAM
  • Single use, single-user tenancy

It looks like so:

Two paradigms -- the old and the new

Two paradigms -- the old and the new

These are fundamentally different ways of thinking about not just distributed systems but computers themselves.  (We can talk about the CAP Theorem and cloud vs. enterprise all day.)

From a software engineering approach, the applications themselves are architected to be:

  • Single Purpose
  • Single Use
  • Largely stateless
  • Provide 1 or maybe 2 coherent and complete API or worker interfaces
  • Herded by a queuing system or herder like Zookeeper

The new term floating around for this is microservices -- tiny, single serving services running on tiny machines which, through coherent API layers, load balancing, storage, and worker coordination, provide one complete system.   Work is subdivided up across herds of microservices to be:

  • Fault tolerant
  • Understanding of network latency
  • Horizontally scaleable (within tolerance of database connections)
  • Auto-scaleable
  • Small
  • Light
  • Single-serving
  • Mostly asynchronous

And from a DevOps point of view, microservices also lend themselves to partial upgrades of single systems, continuous integration, rolling restarts, and management by herds.  It also brings up all sorts of new problems from the same point of view:  how does one monitor?  How does one log?  How does one deal with dead VMs?  How does one build a build and release system with automated configuration that works smoothly with this architecture?  All good questions, and lots of talk.  Some answers.

This is harder to design that in looks since it means going into a software project going "okay now how do I break this up?"  How does one deal with service death?  Network outages?  Network issues?  Communication between these microservices?  How does one build a complete and coherent API layer to tie the services together?

This is good stuff and all new applications going forward should be designed with tiny, flexible services in mind.  As AWS and Google Compute Cloud grow and the big companies all get into building self-service clouds, the future of computer architecture is huge distributed clouds over multiple datacenters built out of microservices and held together with coherent APIs and GTM.