Project Butterfly isn't quite to LEGOs yet. It's more at the DUPLO stage.
Having picked out an operating system, a programming language, some technologies, and a starting database, the high level architecture of Project Butterfly looks like this:
Some call-outs and considerations here because I jumped a few hoops to get to a first overall end state and moved some pieces around from a traditional simple LAMP stack.
Any application with customer data must have some sort of customer data management. Even my little blog has a management interface. Separating the administrative application from the main application at this super early stage is both a security feature and a way to avoid refactoring. Sometimes it's okay to have the administrative interface be part of the stack like the way Wordpress does it but for most commercial applications, it's an excellent way to get hacked. Friends don't let. friends ship their admin interface anywhere close to the network edge.
The admin interface will only need caching if it starts being part of a bigger visualization and real time analytics suite... which it might! But for now it will only perform some simple functions.
Breaking Front End and Middleware
At this early point, it feels like overkill and over-engineering to break apart the front end and force it to call through a REST library to a middleware service -- especially as there will be 1 node.js process of each running on different ports. If only a handful of people use the site, and its easier to build it with one Node.JS + Express stack, isn't that faster to get to completion. Well yes but...
- Breaking apart the front end and middleware tier is always a good idea. At the very least, it allows two people to work on the application at the same time -- important for when the application grows.
- This is modularity from the architecture layer. Now, if I don't like node.js as a front end and would rather run PHP on php5-fpm and build out more of a CMS that consumes my already built node.js API, I can and will lose little. And if I decide to replace MySQL with Cassandra or MongoDB, the front end stays the same while the middle tier changes.
- Modularity also means containing application rights and responsibilities -- important when putting the system into security groups later. The front end is responsible for display and session management. The middleware is responsible for database calls.
- Forcing the front end to go through a REST layer for its database calls protects it from SQL injection.
- At some point in the middle of the application's lifecycle, likely toward the end when trying to get it to scale and realizing that the giant monolithic fat app that does all the display and all the database calls and all the caching doesn't scale at all, this first necessary refactor is already completed.
- Front ends and middleware systems that do work scale horizontally at different rates.
- In a complex system, other systems may want to integrate with this piece.
This won't scale to millions of people. But we're thinking about scaling to millions of people early in the process. That RESTful interface may rapidly turn into a worker instead, or become just a front end to workers -- because distributed workers are ultimately how we get to the Big Numbers.