Roger Qiu

Configuration divergence is a result of change entropy in an environment that doesn't enforce referential integrity between loosely-coupled state or "state-at-a-distance". Simultaneously this allows the anti-pattern of action at a distance. This process gives us a system that becomes more and more opaque as time continues, and we lose referential transparency on the entire system. The result of course is a repeated cycle of entropic reduction projects (often called "refactoring" or "rewriting") where consultants are hired to clean up technical debt. Occasionally these projects will suffer from the second system effect. With this in mind, the significant difference between data…

Matrix AI is maintaining a list of Orchestration platforms, DevOps tools, Distributed Schedulers, Cloud Operating Systems, almost all of them in one place! Check it out at http://datacenteroperatingsystem.io/ There's a lot more out there, so feel free to submit a pull-request. All of these competing solutions arise from different philosophies and are at different levels of maturity. If you're deciding which technology to go with, it's bit too early right now to pick a winner. However it will be prudent to keep track of all these projects including ours Matrix AI and to get your engineers test them…

How do you manage service oriented architecure (SOA) when your services are at differing levels of maturity? We encountered that question at the CTO summit in Sydney. It turns out, that we can use the lens architecture as model to understand this problem and what the possible solutions are: The situation is that there are 3 microservices, each at differing levels of maturity. From the client's perspective, it just sees a single state machine with a single input and output. The entry service would have to interact with its dependent microservice, and that dependent microservice would have its own dependency.…

It's an exciting time in the infrastructure orchestration/automation space. The technologies are moving fast, but here are the interesting things we've observed in the past week: Docker & Rocket Controversy Docker has moving ahead extending their platform with tools such as: Docker Machine - Basically configuration management (CM) for Machines Docker Swarm - Basically a coordination protocol between Docker daemons Docker Compose - Basically a CM focused just on Docker apps The technologies involved aren't new, but it seems that they are focused on streamlining and constraining the tool focus to be solely on their own platform. After all,…

There is a problem that is common to databases, user interface, programming languages and pretty much everything that involves state in more than 1 location. It's called the "view update problem". Or in a more general sense: (Bidirectional) Model Transformation. In the functional programming world, this is modelled via a concept called Lenses. It turns out that this elegant idea, is a great way to model distributed system architectures and the pattern appears in many contexts. Simply put, bidirectional transformation occurs whenever you want consistency between tranformations of state in one location to transformations of state in another location. In…

I've been researching different ways of constructing the Forge Architect DSL. There are tons of different tools and different algorithms used to lex, tokenise, and evaluate context free languages: LL LR (LALR and SLR) Recursive Descent PEG Earley I found OMeta early on, and after reading Alessandro Warth's PHD dissertation, it appeared that OMeta was well suited to prototyping new domain specific languages. OMeta is based on Parsing Expression Grammars (PEG) that has been extended to handle some of the limitations that the original specification of PEGs had. For example it supports left recursion easily through a simple seed parsing…

Reification and abstraction are 2 fundamental concepts on computer programming and how humans communicate in general. These 2 concepts are also guiding the development of the Forge Architect orchestration language. Abstraction is the process of creating an construct that captures the essence of that which is concrete. It can be done via identifying generic patterns that are shared across all of the concrete instances. For example, inventing the abstract construct of Animal derived from the specific Cat and Dog concrete instances that you observe in real life. Abstraction is how people create and communicate high level concepts and derive high…

The term configuration management has a long history beyond software development. When applied to software, people usually refer to the implementations of: CFEngine Puppet Chef Ansible Salt Docker Kubernetes Terraform Fleet And lots more... In a nutshell, configuration management in software refers the process of configuring the servers, and then writing the intructions that allow your software to deploy and run. The instructions also encode more configuration options like environment variables, keys and passwords that will be used by the software. In some cases, configuration management involves installing static packages/libraries, in other cases it's installing a app and then…

At the beginning of April 2014, we began with an idea for some sort of futuristic platform as a service designed for microservices also known as "service oriented architecture". Our first iteration at describing ourselves was: Matrix - Realtime Adaptive Orchestration AI operating on Cellular Service Oriented Automata Architecture This eventually became (mainly because of our increasing knowledge of cellular automata): Realtime Adaptive Orchestration AI operating on Cellular Automata inspired Service Oriented Architecture Still nobody knew what we were talking about. Therefore, upon the national Australian iAwards pitching event, we described ourselves: An operating system for the cloud and internet…