Complex Distributed Application or Complex Orchestration of Microservices?

The question is, what’s the difference?

That assumes though that there is a fundamental difference, and with that assumption I beg to disagree. The number of failures I’ve seen over the past few years trying to orchestrate microservices on the assumption it’s basically easier than writing a distributed application tends to reinforce that disagreement, particularly when the reasons for the failures are looked into.

The similarities between the two are, or at least should be, obvious. In a distributed application, different node types have different responsibilities. This simplifies each node type relative to a monolithic application; likewise, in an orchestration of microservices each service is responsible for one job, simplifying it even more radically than most distributed application nodes. But in orchestrating those simple services the complexity avoided in the service rears its ugly head even higher than the complexities avoided in each node type of a distributed application, and it does so for two reasons:

1. It hasn’t usually been considered sufficiently by the architects/developers.

2. The almost naïve simplicity of ReST interfaces and HTTP protocols don’t provide any functionality to accomplish it.

First, let’s get rid of one long standing myth, that of a stateless application. As one former manager put it bluntly: “an application is by definition a fucking state machine.” The usual counterexample, the WWW, works because state is maintained by a combination of the underlying internet infrastructure and the minds of the users who keep track of where they’ve clicked and whether it took them anywhere useful. Most applications have neither luxury.

Looking at the Actor framework as an example of a distributed application framework, while communication and the means of changing state are very restrictive, reducing the potential for side effects on state changes, structures such as mailboxes, channels and pub/sub constructs allow for the creation of complex state and context interchanges between nodes. I’m differentiating context from state in this situation where state involves the variables a node is responsible for maintaining, while context involves larger picture variables needed for lookups and other activities that are needed in order for the node to have the necessary information to properly maintain its own state.

State and context work in a very similar way in orchestrating microservices, hence the invention of things like Vert.x and Qbit that provide similar constructs to microservices, with the inherent complexity fully intact, but in a stripped down fashion that neither works in a consistent manner with the services themselves, nor works over an interface and protocol combination that provides assistance in some sort of propagation that modifies the context of another service when the state of one service changes.

Of course, orchestrating simple services into useful applications is not as simple as writing simple services individually. By virtue of being useful applications almost inevitably involve some complexity. However it is of course possible. The idea that the simplicity of the services simplifies orchestration is back-asswards. Wherever one area is oversimplified complexity inevitably increases in another. Tools such as Vert.x and Qbit offer some assistance (as on the protocol level do tools such as K8s and Traefik) but at the cost of a severe loss of performance (Vert.x), a lot of manual rote code writing (Qbit), and a much more complex deployment infrastructure (K8s and Traefik) and the resulting need for better devops tools in the latter case, and better architects and developers in the former.

The crossovers between Actor framework implementations such as Quasar or Loom, and microservices orchestrators such as Vert.x and Qbit, accomplished by libraries such as Vert.x Sync and the Qbit or Vert.x eventbus bridge, provide the potential capabilities and performance needed to successfully orchestrate microservices without serious performance degradation, but the complexity of Quasar itself, on top of Vert.x or Qbit or some combination of the two, is a learning curve not many developers I’ve met want to try to climb.