Week #51's Wednesday Evening Training: A practical introduction to Graph Databases and Neo4j

In this week's Wednesday Evening Training, Fred van Nimwegen and I gave an introduction to graph-oriented databases, and Neo4j in particular.

Why a session on this topic?

During our work with on of our customers, Fred and I work on quite large architectures, modeling them in Archimate and UML. Having an architecture model is only step 1, using it in an effective way is really important and is gives much more challenges. Usually, a collection of usefull architecture views are defined to illustrate the architecture from the point of view of the stakeholders. The Archimate standard provides a set of views and the 4+1 architectural view model as well. But when architectures are getting more extensive and more dynamic, you'll more often find yourself needing a powerful way of "searching" and "selecting" relevant elements in an architecture model. Since architectures can be stored in databases (e.g. by using plugins like the Database export/import plugin for Archi), you can query them. Fred and I have done some research on this and found out that querying a relational database isn't that efficient. You can look on an architecture model as a collection of element (types) interconnected with each other. Each architecture element can have a relation of some kind with another element. For example: a service has a assignment relation with an application interface that has a realization relation with an application that can have serving relations with other applications. Finding all services that in some way are influences by infrastructure processes, for example, is not that easy. It's a network that is hard (inefficient) to query on using a relational database using, for example, joins or procedures. This is caused by the concept of a relational database that just isn't suitable for models that look like networks of interconnected elements. Luckily, the Database export/import plugin for Archi also supports Neo4j, a graph database, which seems to be more suitable for this. Graph databases like Neo4j actually provide an excellent way of displaying complex architectures in which there exist many connections between elements.

So what is a graph database? And what is Neo4j?

A graph database is a type of NoSQL database that uses graph theory to store, map and query relationships. This is essentially a collection of nodes and edges. Each node represents an entity and each edge represents a connection or relationship between two nodes. Graph databases are very well suited for analyzing networks, e.g. to mine data from social media.

Neo4j is a graph database management system developed by Neo4j, Inc. Described by its developers as an ACID-compliant transactional database with native graph storage and processing, Neo4j is the most popular graph database according to DB-Engines ranking, and the 22nd most popular database overall.

Back to our Wednesday Evening Training...

This evening, Fred explained us the basics of graph databases, and us how to query networks using the Cypher Query Language. He also showed us prototypes of extensive Archimate models generated using Neo4j. Using Neo4j and three.js (JavaScript library) 3D views of architecture models (networks) were displayed and filtered, showing all kinds of relations between architecture elements. Quite a useful way of gaining insight in a complex architecture!

Of course there was plenty of opportunity for experimenting. We did some handson labs and did walk throughs of some examples and demo's. You can find these in the resources mentioned below.

We'll continue our research on architecture modeling using Neo4j for sure and in upcoming Wednesday Evening Trainings we'll update our community on our proceedings.

Further reading

Do you want to read more on the topics in this post or play around with the technology? Here are some links…

On Graph databases:
What is a Graph Database? https://neo4j.com/developer/graph-database
Graph Databases for Beginners: Why Graph Technology Is the Future: https://neo4j.com/blog/why-graph-databases-are-the-future
Graph database: https://en.wikipedia.org/wiki/Graph_database

On Neo4j:

Neo4j (GitHub repo): https://github.com/neo4j/neo4j
Neo4j homepage: https://neo4j.com/
Cypher Query Language: https://neo4j.com/developer/cypher-query-language/
Fred's GitHub repo (explanation, examples, in Dutch): https://gitlab.com/fredvn-woensdagavond/neo4j-2018-12
Learn Cypher in minutes: https://learnxinyminutes.com/docs/cypher
Neo4j Sandbox (handson labs, built-in guides and sample datasets for popular use cases): https://neo4j.com/sandbox-v2

Examples:
The Graph of Thrones [Season 7 Contest]: https://neo4j.com/blog/graph-of-thrones/
Dataset: Movie Database: https://neo4j.com/developer/movie-database/#_the_dataset
Exploring a UK Open Government Dataset with Neo4j: https://leanjavaengineering.wordpress.com/2015/04/10/exploring-a-uk-open-government-dataset-with-neo4j

On related topics:
4+1 architectural view model: https://en.wikipedia.org/wiki/4%2B1_architectural_view_model
Database export/import plugin that store Archi models in a central database repository: https://github.com/archi-contribs/database-plugin
threejs (cross-browser JavaScript library and Application Programming Interface (API) used to create and display animated 3D computer graphics): https://threejs.org/

The next Wednesday Evening Training will be in 2019

January 9th 2019 will be our first Wednesday Evening Training in 2019. We'll start 2019 with an IoT "klusavond": continuing our projects on Arduino, Raspberry Pi and Whitty Cloud. Two of our IoT experts, Aishwarya Dhall and Luc Slutter, will share their knowledge and experience with us. We'll take a deep dive in interrupts and we'll get an introduction into the Domoticz lightweight Home Automation System.

Past Wednesday Evening Trainings on all topics

You 'll find post of previous sessions here: https://www.linkedin.com/search/results/content/?keywords=%23wednesdayeveningtraining