Polykey - Distributed Secret Sharing

The Matrix AI team has been developing Polykey, a distributed peer-to-peer secret sharing system. It is intended to manage secrets, passwords, API keys for both humans and machines. Many secret management systems have been designed either only for humans, or only for machines. We think this is unnecessary and intend…

Developing with Nix (C, JavaScript, Python, Haskell, Emscripten, PHP)

Nix, NixOS and NixPkgs allows us to to create project-specific development environments with project-specific dependencies (this usually means things like a C project, or a Python project... etc). The way this is done is different for every language community within the Nix ecosystem. The most well developed patterns would be…

Intro to Nix Channels and Reproducible NixOS Environment

This introduction assumes you have played with NixOS a bit, you know about content addressability and why it is important, and how Git repositories represent a distributed content addressed storage system. Git and Github is used as the source control for all of NixOS and NixPkgs. Both NixOS and NixPkgs…

Configuration Divergence

Configuration divergence is a result of change entropy in an environment that doesn't enforce referential integrity between loosely-coupled state or "state-at-a-distance". Simultaneously this allows the anti-pattern of action at a distance. This phenomenon is also called "Connascence" and I have also called this "meta-coupling"…

Reproducible Builds vs Semantic Versioning

Today, the most common way for developers to address a dependency is through the use of semantic versioning. This is where you create project A that depends on B at 1.2.3, where 1 is the major version, 2 is the minor version and 3 is the patch version.…

REST Composition & Aggregation

REST is a protocol used commonly for web APIs. It generally means each resource is identified using a a URL path like /resource1. Each of the resource end points supports a RESTful operations that is: GET POST PUT PATCH DELETE. A common problem occurs is when the client needs to…

Declarative Stateless Migrations in Databases

Ever since we read Domen Kožar's thoughts on how Nix and NixOS is a superior configuration management system because it features: Declarative Semantics Stateless Deterministic (pure) Avoids dependency hell Idempotent Atomic/Transactional Reversible or Rollbacks We've been thinking about how to apply the Nix philosophy to live database migrations. (Specifically…

Data Center Operating Systems

Matrix AI is maintaining a list of Orchestration platforms, DevOps tools, Distributed Schedulers, Cloud Operating Systems, almost all of them in one place! Check it out at http://datacenteroperatingsystem.io/ There's a lot more out there, so feel free to submit a pull-request. All of these competing solutions arise from…

Functionally Composing Service Oriented Architecture

How do you manage service oriented architecure (SOA) when your services are at differing levels of maturity? We encountered that question at the CTO summit in Sydney. It turns out, that we can use the lens architecture as model to understand this problem and what the possible solutions are: The…

Docker, Rockets, Mesos and Atlas

It's an exciting time in the infrastructure orchestration/automation space. The technologies are moving fast, but here are the interesting things we've observed in the past week: Docker & Rocket Controversy Docker has moving ahead extending their platform with tools such as: Docker Machine - Basically configuration management (CM) for…