Today, the most common way for developers to address a dependency is through the use of semantic versioning. This is where you create project A that depends on B at 1.2.3, where 1 is the major version, 2 is the minor version and 3 is the patch version. Accordingly, the rules of following semantic versioning means: Major versions change when you make incompatible API changes. Minor versions change when you add functionality in a backwards-compatible manner. Patch versions change when you make backwards-compatible bug fixes. Theoretically you should be able to update patch and minor versions of dependencies…

REST is a protocol used commonly for web APIs. It generally means each resource is identified using a a URL path like /resource1. Each of the resource end points supports a RESTful operations that is: GET POST PUT PATCH DELETE. A common problem occurs is when the client needs to aggregate resources from a RESTful API, or to compose together operations on multiple resources. This is called the Composition & Aggregation problem. There's actually a number of distinct usecases here: The client wants to run a single operation on aggregate multiple resources: GET2 /resource1, /resource2 or POST2 /resource1, /resource2. (Applicative)…

Ever since we read Domen Ko┼żar's thoughts on how Nix and NixOS is a superior configuration management system because it features: Declarative Semantics Stateless Deterministic (pure) Avoids dependency hell Idempotent Atomic/Transactional Reversible or Rollbacks We've been thinking about how to apply the Nix philosophy to live database migrations. (Specifically relational SQL databases.) Our motivation is due to problems faced by current database migration solutions. Most solutions do not feature "desired-state" configuration. Instead the migrations are coded in imperative manner, coding the explicit actions to take in order to migrate schema or data. The migration can fail which means they…

Matrix AI is maintaining a list of Orchestration platforms, DevOps tools, Distributed Schedulers, Cloud Operating Systems, almost all of them in one place! Check it out at http://datacenteroperatingsystem.io/ There's a lot more out there, so feel free to submit a pull-request. All of these competing solutions arise from different philosophies and are at different levels of maturity. If you're deciding which technology to go with, it's bit too early right now to pick a winner. However it will be prudent to keep track of all these projects including ours Matrix AI and to get your engineers test them…

How do you manage service oriented architecure (SOA) when your services are at differing levels of maturity? We encountered that question at the CTO summit in Sydney. It turns out, that we can use the lens architecture as model to understand this problem and what the possible solutions are: The situation is that there are 3 microservices, each at differing levels of maturity. From the client's perspective, it just sees a single state machine with a single input and output. The entry service would have to interact with its dependent microservice, and that dependent microservice would have its own dependency.…

It's an exciting time in the infrastructure orchestration/automation space. The technologies are moving fast, but here are the interesting things we've observed in the past week: Docker & Rocket Controversy Docker has moving ahead extending their platform with tools such as: Docker Machine - Basically configuration management (CM) for Machines Docker Swarm - Basically a coordination protocol between Docker daemons Docker Compose - Basically a CM focused just on Docker apps The technologies involved aren't new, but it seems that they are focused on streamlining and constraining the tool focus to be solely on their own platform. After all,…

There is a problem that is common to databases, user interface, programming languages and pretty much everything that involves state in more than 1 location. It's called the "view update problem". Or in a more general sense: (Bidirectional) Model Transformation. In the functional programming world, this is modelled via a concept called Lenses. It turns out that this elegant idea, is a great way to model distributed system architectures and the pattern appears in many contexts. Simply put, bidirectional transformation occurs whenever you want consistency between tranformations of state in one location to transformations of state in another location. In…

I've been researching different ways of constructing the Forge Architect DSL. There are tons of different tools and different algorithms used to lex, tokenise, and evaluate context free languages: LL LR (LALR and SLR) Recursive Descent PEG Earley I found OMeta early on, and after reading Alessandro Warth's PHD dissertation, it appeared that OMeta was well suited to prototyping new domain specific languages. OMeta is based on Parsing Expression Grammars (PEG) that has been extended to handle some of the limitations that the original specification of PEGs had. For example it supports left recursion easily through a simple seed parsing…

Reification and abstraction are 2 fundamental concepts on computer programming and how humans communicate in general. These 2 concepts are also guiding the development of the Forge Architect orchestration language. Abstraction is the process of creating an construct that captures the essence of that which is concrete. It can be done via identifying generic patterns that are shared across all of the concrete instances. For example, inventing the abstract construct of Animal derived from the specific Cat and Dog concrete instances that you observe in real life. Abstraction is how people create and communicate high level concepts and derive high…

The term configuration management has a long history beyond software development. When applied to software, people usually refer to the implementations of: CFEngine Puppet Chef Ansible Salt Docker Kubernetes Terraform Fleet And lots more... In a nutshell, configuration management in software refers the process of configuring the servers, and then writing the intructions that allow your software to deploy and run. The instructions also encode more configuration options like environment variables, keys and passwords that will be used by the software. In some cases, configuration management involves installing static packages/libraries, in other cases it's installing a app and then…