Reification and abstraction are 2 fundamental concepts on computer programming and how humans communicate in general. These 2 concepts are also guiding the development of the Forge Architect orchestration language.
Abstraction is the process of creating an construct that captures the essence of that which is concrete. It can be done
via identifying generic patterns that are shared across all of the concrete instances. For example, inventing the
abstract construct of Animal derived from the specific Cat and Dog concrete instances that you observe in real
life. Abstraction is how people create and communicate high level concepts and derive high level relationships from low
level observation. It provides the ability for humans to represent knowledge and reason.
Reification (sometimes called refinement) is slightly less well known concept, this is the opposite of abstraction. It
is the process of turning what is abstract into something that is concrete. For example manufacturing a Car from a Car
blueprint. Or going from something that generic to something that is more specific like from Person to PersonWithHat
. Or encoding an abstract implicit relationship in a SQL database into first class primitive entities that can be
manipulated.
It should be clear that reification and abstraction exist on a continuum axis of concept representation . Depending on where you sit on the continuum, you go up the abstraction ladder, or go down the reification slide.
Every programming language balances between the axis of abstraction and reification in the whole and in their various parts. Whether you go up in terms of abstraction or go down in terms of reification, you make tradeoffs that are relevant in the signal/noise ratio in your communicative expressions. Similar ideas occur in the discussions of the Blub idea by Paul Graham. Being more abstract in your expressions only makes sense if your context allows you to ignore the low level operations. Being more reified in your expressions only makes sense if your context involves solving problems at the low level.
The Matrix Forge Architect language is intended to abstract the real infrastructure operation into a more abstract language model, while also reifying the implicit infrastructure expectations into concrete explicit specifications that can be reasoned about.
Abstraction gives us more power to express higher level concepts with less noise (when in higher level contexts), whereas reification makes things that are implicit more explicit, thus resolving in greater clarity, and allow us to concretely reason about what was once very abstract (in natural languages you might hear the term "wishy washy").
Configuration management systems today face 2 problems:
They mostly do not allow us to succinctly (with high signal to noise ratio) express high level concepts in infrastructural orchestration. Thus they lack abstraction. (when they do, they are often naively implemented, and not sufficiently abstract nor composable, similar to a reification fallacy in natural languages)
They often do not explicitly give control over the implicit infrastructural relationships that we expect to exist. Thus they lack reification.
Example of not enough abstraction:
The utilisation of IP + ports to address dependencies is a form of incidental complexity. These should be abstracted to the essential chracteristic of a dependency, which is the API of the dependency, not the geographically allocated IP + port.
Example of not enough reification:
In some cases, there isn't enough control over a particular factor of orchestration, and this is left implicitly managed by manual commands, or in other scripts somewhere else.
Let's imagine the dependency relationship a Wordpress app and a MySQL database. The implementation of this factor is written in PHP somewhere. This factor should instead be abstracted out of the PHP app, but it is actually infact reified in the configuration management language.
This is because according to the configuration management's point of view, you may be writing some configuration snippet that implicitly relies on the expectation that there is a dependency between the Wordpress and Mysql. However this relationship is not actually guaranteed by the language, instead it's simply expected to exist, and the actual guarantees are somewhere else, also known as a spooky action at a distance.
To a new developer not associated with your system, reading the configuration management language does not give the person an understanding or the ability the reason about the relationship between the wordpress and the mysql, it's simply implicit knowledge that they would not necessarily grok until they start inspecting the rest of your infrastructure.
Therefore, such a factor should be reified from the point of view of the configuration language, but abstracted from the point of view from your PHP app.
Thus any control system such as Forge Architect needs to balance between abstraction and reification.
Similar Ideas in other Areas:
- LessWrong.com SotW Be Specific
- Tutorial explaining the Ladder of Abstraction by Bret Victor (a former Apple designer among other things!)
- These slides which demonstrate programming languages each have their appropriate level of abstraction:
This post is subject to further update we uncover more use cases and factors that can be reified or abstracted.