(I'll be updating this intro with additional explanations & insights, so check this post regularly)
In my work as an architect I'm dealing with quite large architectures, modeling them in Archimate and UML. Having an architecture model is only step 1, using it in an effective way is really important and is gives much more challenges. Usually, a collection of usefull architecture views are defined to illustrate the architecture from the point of view of the stakeholders. The Archimate standard provides a set of views and the 4+1 architectural view model as well. But when architectures are getting more extensive and more dynamic, you'll more often find yourself needing a powerful way of "searching" and "selecting" relevant elements in an architecture model. For example, when determining impact of architectural changes, this may be the case. Since architectures can be stored in databases (e.g. by using plugins like the Database export/import plugin for Archi), you should be able to query them, hence creating dynamic architecture views.
Querying a relational database
I have done some research on this and found out that querying a relational database isn't that efficient. You can look on an architecture model as a collection of element (types) interconnected with each other. Each architecture element can have a relation of some kind with another element. For example: a service has a assignment relation with an application interface that has a realization relation with an application that can have serving relations with other applications. Finding all services that in some way are influences by infrastructure processes, for example, is not that easy. An architecture is actually a network that is hard (inefficient) to query on using a relational database using, for example, joins or procedures. This is caused by the concept of a relational database that just isn't suitable for models that look like networks of interconnected elements. Luckily, the Database export/import plugin for Archi also supports Neo4j, a graph database, which seems to be more suitable for this. Graph databases like Neo4j actually provide an excellent way of displaying complex architectures in which there exist many connections between elements.
So what is a graph database? And what is Neo4j?
A graph database is a type of NoSQL database that uses graph theory to store, map and query relationships. This is essentially a collection of nodes and edges that needs to be queried. Each node represents an entity and each edge represents a connection or relationship between two nodes. Graph databases are very well suited for analyzing networks, e.g. to mine data from social media.
Neo4j is a graph database management system. It can be described as an ACID-compliant transactional database with native graph storage and processing.
Neo4j databases can be queried by using a special query language: Cypher.
The Cypher Query Language
Cypher is a declarative language for expressive and efficient querying and maintaining a property graph. It's based on the Property Graph Model, based on elements of nodes and edges (relationships between elements) and labels and properties that can be added to them.
An example query for Actors in one of the example Neo4j databases available is:
MATCH (nicole:Actor {name: 'Nicole Kidman'})-[:ACTED_IN]->(movie:Movie) WHERE movie.year < $yearParameter RETURN movie
You'll find some nice instructive video's on Graph Databases on YouTube...
Intro to Graph Databases Episode #5 - Cypher, the Graph Query Language
More in this series:
Intro to Graph Databases Episode #1 - Evolution of DBs
Intro to Graph Databases Episode #2 - Properties of Graph DBs & Use Cases
Intro to Graph Databases Episode #3 - Property Graph Model
Intro to Graph Databases Episode #4 - (RDBMS+SQL) to (Graphs+Cypher)
Intro to Graph Databases Episode #5 - Cypher, the Graph Query Language
Intro to Graph Databases Episode #6 - Continuing with Cypher
In this week's Wednesday Evening Training, Fred van Nimwegen and I gave an introduction to graph-oriented databases, and Neo4j in particular.
Why a session on this topic?
During our work with on of our customers, Fred and I work on quite large architectures, modeling them in Archimate and UML. Having an architecture model is only step 1, using it in an effective way is really important and is gives much more challenges. Usually, a collection of usefull architecture views are defined to illustrate the architecture from the point of view of the stakeholders. The Archimate standard provides a set of views and the 4+1 architectural view model as well. But when architectures are getting more extensive and more dynamic, you'll more often find yourself needing a powerful way of "searching" and "selecting" relevant elements in an architecture model. Since architectures can be stored in databases (e.g. by using plugins like the Database export/import plugin for Archi), you can query them. Fred and I have done some research on this and found out that querying a relational database isn't that efficient. You can look on an architecture model as a collection of element (types) interconnected with each other. Each architecture element can have a relation of some kind with another element. For example: a service has a assignment relation with an application interface that has a realization relation with an application that can have serving relations with other applications. Finding all services that in some way are influences by infrastructure processes, for example, is not that easy. It's a network that is hard (inefficient) to query on using a relational database using, for example, joins or procedures. This is caused by the concept of a relational database that just isn't suitable for models that look like networks of interconnected elements. Luckily, the Database export/import plugin for Archi also supports Neo4j, a graph database, which seems to be more suitable for this. Graph databases like Neo4j actually provide an excellent way of displaying complex architectures in which there exist many connections between elements.
So what is a graph database? And what is Neo4j?
A graph database is a type of NoSQL database that uses graph theory to store, map and query relationships. This is essentially a collection of nodes and edges. Each node represents an entity and each edge represents a connection or relationship between two nodes. Graph databases are very well suited for analyzing networks, e.g. to mine data from social media.
Neo4j is a graph database management system developed by Neo4j, Inc. Described by its developers as an ACID-compliant transactional database with native graph storage and processing, Neo4j is the most popular graph database according to DB-Engines ranking, and the 22nd most popular database overall.
Back to our Wednesday Evening Training...
This evening, Fred explained us the basics of graph databases, and us how to query networks using the Cypher Query Language. He also showed us prototypes of extensive Archimate models generated using Neo4j. Using Neo4j and three.js (JavaScript library) 3D views of architecture models (networks) were displayed and filtered, showing all kinds of relations between architecture elements. Quite a useful way of gaining insight in a complex architecture!
Of course there was plenty of opportunity for experimenting. We did some handson labs and did walk throughs of some examples and demo's. You can find these in the resources mentioned below.
We'll continue our research on architecture modeling using Neo4j for sure and in upcoming Wednesday Evening Trainings we'll update our community on our proceedings.
Further reading
Do you want to read more on the topics in this post or play around with the technology? Here are some links…
The next Wednesday Evening Training will be in 2019
January 9th 2019 will be our first Wednesday Evening Training in 2019. We'll start 2019 with an IoT "klusavond": continuing our projects on Arduino, Raspberry Pi and Whitty Cloud. Two of our IoT experts, Aishwarya Dhall and Luc Slutter, will share their knowledge and experience with us. We'll take a deep dive in interrupts and we'll get an introduction into the Domoticz lightweight Home Automation System.
In this week's Wednesday Evening Training we had some great demo's and discussions on our individual projects on IoT and 3D modelling.
A lot of IoT
Aish showed us her giant LED cube (10x10x10 LED's, that's a lot of soldering!) and explained to us how she managed to build and run it. We discussed powering such a large collection of LED's and how the software was designed. Such a bug LED cube offers plenty of applications, including running 3D animations and games within the cube. We've started building a graphics library for basic LED handling that is going to be a great help in building such applications. Aish also showed us her multi sensor project; a test platform on which all sorts of sensors are installed. We discusses the various sensor types, how to connect and use them and how (combined) measurements can best be done. Interesting stuff! One of the topics we also discussed is the way which the sensors are read in the software. That can be done in a loop but also using interrupts (low level event handlers). The latter method is faster, less error prone and cleaner. We'll be investigating this technique in the upcoming Wednesday Evening Training IoT sessions.
Archimate architecture models in BabylonJS 3D
This week's session, I gave give a demo of my BabylonJS prototype in which an Archimate model view (using the free Archi editor) is converted into a BabylonJS 3D model. For complex architectures, 3D modelling may give more insight than plain 2D models. Worth doing some experiments to see whether this is a good approach. The Technical Proof of Concept is running great. It uses MySQL as a central database and a simple website with some Web Workers to gather model data, generating and continuously updating a JavaScript Archimate model and a derived Babylon 3D model from this in the background. Our colleague Fred van Nimwegen also joined my project and he had experimented with Neo4j in which he stored the Archimate model. Neo4j is a graph database (while MySQL is a relational database). Graph databases have some advantages over relational databases, mainly when querying data collections that naturally resemble networks. And that is exactly what an architectural model resembles: a network of related nodes. Moreover: modern architectures are getting more dynamic and will change over time more rapidly. When analyzing an architecture model to answer architectural questions like "what will happen when I change this?", an efficient query language and dito database are very convenient. We've planned a Wednesday Evening Training session on Graph databases and Neo4j in november. Fred will then share his knowledge gained with us.
Further reading
Do you want to read more on the topics in this post? Here are some links…
Next week, we'll have the intro on Progressive Web Apps (PWA). This a hot topic in the world of web development and its only growing further in popularity. It is a relatively new way to develop applications for the mobile phone, combining the best of the web and apps. We will cover benefits using a PWA instead of traditional hybrid/native solutions, the basics and possibilities of a PWA, show some examples in functionality and give an example of how you can get started building your own PWA’s quickly. Yes, there will also be a handson lab. In later sessions we expect to cover topics like push notifications, background refresh, et cetera.
