12/25/18

Week #51's Wednesday Evening Training: A practical introduction to Graph Databases and Neo4j

In this week's Wednesday Evening Training, Fred van Nimwegen and I gave an introduction to graph-oriented databases, and Neo4j in particular.



Why a session on this topic?

During our work with on of our customers, Fred and I work on quite large architectures, modeling them in Archimate and UML. Having an architecture model is only step 1, using it in an effective way is really important and is gives much more challenges. Usually, a collection of usefull architecture views are defined to illustrate the architecture from the point of view of the stakeholders. The Archimate standard provides a set of views and the 4+1 architectural view model as well. But when architectures are getting more extensive and more dynamic, you'll more often find yourself needing a powerful way of "searching" and "selecting" relevant elements in an architecture model. Since architectures can be stored in databases (e.g. by using plugins like the Database export/import plugin for Archi), you can query them. Fred and I have done some research on this and found out that querying a relational database isn't that efficient. You can look on an architecture model as a collection of element (types) interconnected with each other. Each architecture element can have a relation of some kind with another element. For example: a service has a assignment relation with an application interface that has a realization relation with an application that can have serving relations with other applications. Finding all services that in some way are influences by infrastructure processes, for example, is not that easy. It's a network that is hard (inefficient) to query on using a relational database using, for example, joins or procedures. This is caused by the concept of a relational database that just isn't suitable for models that look like networks of interconnected elements. Luckily, the Database export/import plugin for Archi also supports Neo4j, a graph database, which seems to be more suitable for this. Graph databases like Neo4j actually provide an excellent way of displaying complex architectures in which there exist many connections between elements.


So what is a graph database? And what is Neo4j?

A graph database is a type of NoSQL database that uses graph theory to store, map and query relationships. This is essentially a collection of nodes and edges. Each node represents an entity and each edge represents a connection or relationship between two nodes. Graph databases are very well suited for analyzing networks, e.g. to mine data from social media.

Neo4j is a graph database management system developed by Neo4j, Inc. Described by its developers as an ACID-compliant transactional database with native graph storage and processing, Neo4j is the most popular graph database according to DB-Engines ranking, and the 22nd most popular database overall.

Back to our Wednesday Evening Training...

This evening, Fred explained us the basics of graph databases, and us how to query networks using the Cypher Query Language. He also showed us prototypes of extensive Archimate models generated using Neo4j. Using Neo4j and three.js (JavaScript library) 3D views of architecture models (networks) were displayed and filtered, showing all kinds of relations between architecture elements. Quite a useful way of gaining insight in a complex architecture!

Of course there was plenty of opportunity for experimenting. We did some handson labs and did walk throughs of some examples and demo's. You can find these in the resources mentioned below.

We'll continue our research on architecture modeling using Neo4j for sure and in upcoming Wednesday Evening Trainings we'll update our community on our proceedings.

Further reading

Do you want to read more on the topics in this post or play around with the technology? Here are some links…

On Graph databases:
What is a Graph Database? https://neo4j.com/developer/graph-database
Graph Databases for Beginners: Why Graph Technology Is the Future: https://neo4j.com/blog/why-graph-databases-are-the-future
Graph database: https://en.wikipedia.org/wiki/Graph_database

On Neo4j:
Neo4j (GitHub repo): https://github.com/neo4j/neo4j
Neo4j homepage: https://neo4j.com/
Cypher Query Language: https://neo4j.com/developer/cypher-query-language/
Fred's GitHub repo (explanation, examples, in Dutch): https://gitlab.com/fredvn-woensdagavond/neo4j-2018-12
Learn Cypher in minutes: https://learnxinyminutes.com/docs/cypher
Neo4j Sandbox (handson labs, built-in guides and sample datasets for popular use cases): https://neo4j.com/sandbox-v2

Examples:
The Graph of Thrones [Season 7 Contest]: https://neo4j.com/blog/graph-of-thrones/
Dataset: Movie Database: https://neo4j.com/developer/movie-database/#_the_dataset
Exploring a UK Open Government Dataset with Neo4j: https://leanjavaengineering.wordpress.com/2015/04/10/exploring-a-uk-open-government-dataset-with-neo4j

On related topics:
4+1 architectural view model: https://en.wikipedia.org/wiki/4%2B1_architectural_view_model
Database export/import plugin that store Archi models in a central database repository: https://github.com/archi-contribs/database-plugin
threejs (cross-browser JavaScript library and Application Programming Interface (API) used to create and display animated 3D computer graphics): https://threejs.org/

The next Wednesday Evening Training will be in 2019

January 9th 2019 will be our first Wednesday Evening Training in 2019. We'll start 2019 with an IoT "klusavond": continuing our projects on Arduino, Raspberry Pi and Whitty Cloud. Two of our IoT experts, Aishwarya Dhall and Luc Slutter, will share their knowledge and experience with us. We'll take a deep dive in interrupts and we'll get an introduction into the Domoticz lightweight Home Automation System.

Past Wednesday Evening Trainings on all topics

You 'll find post of previous sessions here: https://www.linkedin.com/search/results/content/?keywords=%23wednesdayeveningtraining



12/17/18

The Wednesday Evening Training: how and why does it work?

Our Wednesday Evening Training in 2018: 50 evenings, over 35 topics on innovative technology, frameworks, architecture, infrastructure, graphics & 3D modeling and Quantum computing in Capgemini’s CoZone.

We learned a lot and we had a lot of fun!

That was 2018, but the Wednesday Evening Training has been running for quite some years now... so how did we do that?

In this blogpost I'll try to explain this by our "rules of the road" that we've stuck by the past years and seem to work really well.



Rule #1: Offer many learning styles and experiment mixing these

Everyone has a personal learning style: the way he or she learns most effectively. As a child we all learned a lot by playing, by experimenting, at our own pace, in our own way. As we got older, the way of learning was increasingly restricted; playing was replaced by fixed learning rituals and focussed on the individual. We simply forgot to learn by playing. In our Wednesday Evening Trainings we offer a platform on which everyone can learn, experiment and share. We mix different learning styles to offer anyone an effective means to get to know new technologies. And having fun is mandatory.



Rule #2: About content: keep the water flowing

Learning content needs to be fresh. Technology is evolving fast. Therefore, publications become stale and less useful, just like water sitting in a glass for too long. To keep learning content usefull it needs to be constantly refreshed, new content must be added, content must be used and discussed actively. It must flow constantly. Dare to deviate from the beaten track. Use many forms of content, use, share and discuss. A discussion in a demo or walkthrough offers plenty of interesting knowledge. We discovered that even an unfinished prototype or crude idea generates more interest and is more inviting than a fully finished and varnished final product. It generates discussion, interaction, and feeds inquisitiveness and ideas. And that's what the Wednesday Evening Training is all about!



Rule #3: Offer a platform anyone can join, not a closed community

By a platform, I do NOT mean a software platform, an App or any other tool that is considered hot. Such tools are NO guarantee for success. A tool is an empty silo that needs a process to be succesfull, content needs to be refreshed and used continuously. If not, the contents of your silo will grow stale and the silo will deminish.
What I mean by a platform is a process anyone can join, at minimum effort. The Wednesday Evening Training takes place at a fixed day/time, in a fixed inspiring location (our CoZone). Joining is easy, just come visit and start working.



Rule #4: There are no guru's nor newbies

Anyone can learn from anyone. Senior professionals can share deep knowledge and broad experience, but young professionals can suggest new subjects that they have just learned at school, they can ask questions and stimulate discussion. It's not just the seniors that teach the juniors. A platform like the Wednesday Evening Training actively encourages exchange knowledge and connects professionals of all levels. And everybody needs to feel safe asking questions, posing idea's and sharing knowledge.



Rule #5: It's a process, not an event, nor a project, nor a program

An event, learning project and program may offer knowledge and experience, but these have a fixed start and and fixed finish. After finishing, learning stops as well. The Wednesday Evening Training, however, goes on continuously, constantly adding new topics and deepen our knowledge and experience in popular topics. Discussion and hands-on experimentation constantly feeds the need for further exploration. And that feeds our Wednesday Evening Training agenda. We plan for the upcoming months and build in plenty of flexibility to add additional topics. What is paramount is to keep the fire burning.


Rule #6: Do not manage, but do facilitate, support, connect, motivate

It may seem like a nightmare to managers, but there is no clearly defined business case, nor deadlines, nor is there a clear deadline on which some kind of deliverable is produced. The Wednesday Evening Training is fully self organized and self-steering. And it continuously produces results: knowledge added to the communities' knowledge base, experience using technologies and new partnerships between professionals. So how can management be involved? Well, in keeping the fire burning: facilitate, support, motivate people to join and actually reward collaboration.




Of course, these rules of the road are constantly evolving. We regularly evaluate how we work together and how we can improve. And if we find it necessary, we adapt. This way we keep up with new technologies, we deepen our knowledge and experience. And above all, we'll continue to have a lot of fun, each week, in our CoZone.

12/14/18

Week #50's Wednesday Evening Training: Docker, Service Workers and a bit of Augmented Reality

This week's Wednesday Evening Training, we really had a busy evening. We had three topics on the menu: Docker, Service workers and Augmented Reality.


Docker Container Platform

Fast and easy scaling is very important nowadays. You need to respond fast to increasing load and you may want to downscale when possible, decreasing running costs. Virtualization is an effective approach to this. Applying virtualization, you actually use your bare metal infrastructure as Host Machines to run software applications (Virtual Machines) that emulate physical machines (Guest Machines). Virtual Machines use Images (files) of completely installed and configured machines (OS and the applications you want to use) to start a Guest Machine. The Virtual Machines themselves are managed (start/pause/resume/stop/...) by so called Hypervisors. Virtualization drastically increases the speed and ease in which you can scale your server infrastructure, but the required emulation and big image files poses a rather expensive overhead.
In Containerization, like Docker does, software (the applications you want to use) is not emulated, but is run directly on the hard hardware of the host machine in separate Container Runtimes (in short: Cointainers). Container Runtimes are based on Docker images which contain definitions of the software and configuration you want to run in your Container Runtimes. This Containerization approach decreases the overhead drastically compared to Virtualization, increasing the advantage of fast and easy scaling.


In our training, I gave an introduction on the technical architecture of a Dockerized software ecosystem and Bart van Beek showed us the actual code: how to setup a Docker image, how to deploy and how to manage Docker Runtimes. He had prepared some nice examples (which he published in his GitHub repo, see the links at the bottom of this post).

Using Containerization in a Micro Services architecture 

In a Microservices architecture (see also my previous post), Containerization is quite handy when deploying Microservices independently and reliably, and (out)scaling them quickly. The challenges however, in managing (orchestrating) such an ecosystem of running Microservice Container Runtimes, requires more software than "bare" Docker containers can provide. In our Wednesday Evening Training, we also briefly discussed Google's Kubernetes, an open-source container-orchestration system for automating deployment, scaling and management of containerized applications. We'll take a deep dive into Kubernetes in one of our upcoming Wednesday Evening Trainings. I'll be writing more on Microservices, Containerization and related topics in a separate blog post I'll be writing shortly.

Service workers: the new Worker especially suitable for Progressive Web Applications (PWA)

Our second topic of this evening, was introduced by Olaf van Venetie, one of our lead front end software engineers. In my post on last week's Wednesday Evening Training, I already briefly discussed the the Worker types available, of which the Service Worker is the most modern and comprehensive.
This evening, Olaf took us on a deep dive into Service Workers. He had prepared a neat demo and gave a code walk through. We had plenty of opportunity for Q&A and creating our own Service worker (see the hands on lab in the links below).


A bit of augmented Reality with A-FRAME and AR.js

The last topic of the evening was actually based on a nice article and some prototypes of a JavaScript based Augmented Reality framework I found some time ago on the web. The technology is based on an all JavaScript implementation of Augmented Reality using the A-FRAME and AR.js frameworks. A-FRAME is a web framework for building virtual reality experiences and AR.js is an efficient Augmented Reality solution for use on the Web. It runs entirely in a web browser and there is no need to install a special App. Combining A-FRAME and AR.js gives developers a simple way to create Augmented Reality content for the web.

Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the real-world are "augmented" by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory. [source: Wikipedia].

In short: you can put (overlay) objects in a camera view. And that is just what we did this evening.

We used our camera's (webcams, camera's on our mobile phones) in a demo web page to point at a piece of paper with a simple and clear icon on it (a marker) and the AR software displayed a 3D animated object, in our screen, on top of the marker. You can do this quite easily yourself; I've added some links for that below. A simple but impressive little demo, that gave us plenty of thoughts on new Wednesday Evening Trainings. 

To be continued for sure!


 

Further reading

Do you want to read more on the topics in this post or play around with the technology? Here are some links…

On Docker:
Docker (official website): https://www.docker.com
Docker (software), a nice intro on Wikipedia: https://en.wikipedia.org/wiki/Docker_(software)
Play with Docker classroom: https://training.play-with-docker.com/
Play with Docker classroom (specific topics): https://training.play-with-docker.com/alacart
Play with Docker - A simple, interactive and fun playground to learn Docker: https://labs.play-with-docker.com/
Bart van Beek's Docker examples (GitHub repo): https://github.com/babeek94122/wet-demo-docker
Deploy on Kubernetes: https://docs.docker.com/docker-for-mac/kubernetes/
Virtualization: https://en.wikipedia.org/wiki/Virtualization

On Service Workers:
Hands on lab building your first Service worker: https://developers.google.com/web/ilt/pwa/lab-scripting-the-service-worker

On Augmented Reality:
Creating Augmented Reality with AR.js and A-Frame: https://aframe.io/blog/arjs
AR.js  - The Simplest Way to get Cross-Browser Augmented Reality on the Web: https://medium.com/chialab-open-source/ar-js-the-simpliest-way-to-get-cross-browser-ar-on-the-web-8f670dd45462
Efficient Augmented Reality for the Web - 60fps on mobile! (GitHub repo): https://github.com/jeromeetienne/ar.js
AR.js examples (GitHub): https://github.com/stemkoski/AR-Examples
AR.js examples: https://jeromeetienne.github.io/AR.js-docs/misc/EXAMPLES.html
My YouTube channel on Augmented Reality (explanations, demo's, et cetera): https://www.youtube.com/playlist?list=PLSiMhBs48YvWFjL98LCyHLtRetCNM9lnR
Basket ball demo using AR.js and A-FRAME.js - Live demo: http://stemkoski.github.io/AR-Examples
Basket ball demo using AR.js and A-FRAME.js - GitHub repo: https://github.com/stemkoski/AR-Examples
Build and access your first AR webapp in less than 10 minutes using Arjs: https://blog.theodo.fr/2018/09/build-first-ar-web-app-less-10mn/
Red animating block demo: https://hansr.viahetweb.com/ar/t1/

Explanation on the block demo: open the URL above in a browser on your mobile phone. Then point your camera at the marker below. You should see a red animating cube in your mobile's screen, hovering above the marker that you're pointing at.


Next Wednesday Evening Training

Next week, we'll concentrate on Neo4J and graph databases. One of our database and Java experts, Fred van Nimwegen, will guide us through the concepts of graph based databases and will whow us an implementation of such a database.

Past Wednesday Evening Trainings on all topics

You 'll find post of previous sessions here: https://www.linkedin.com/search/results/content/?keywords=%23wednesdayeveningtraining


#wednesdayeveningtraining #capgemini #lifeatcapgemini #docker #containerization #serviceworkers #pws #html5 #ar #augmentedreality



12/6/18

Week #45's Wednesday Evening Training: Microservices - What, why and how?

Tonight it's Sinterklaasavond in Holland, so there is no Wednesday Evening Training. Hence a good time to settle my debt and write a blog article on week #45's Wednesday Evening Training. I had not finished it yet.

Microservices: what are we talking about?

A Microservice is a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of fine grained, loosely coupled services using lightweight protocols. These services can be deployed separately, so that development can be parallelized enabling small autonomous teams to develop, deploy and scale services independently. This allows the architecture of individual services to emerge through continuous refactoring and enables continuous delivery and deployment.

Dividing an application into small pieces also gives additional challenges. Coordination, flow control, integration, monitoring and exception management for example, will have to be supported separately. That is why a Microservice architecture should be provided with Microservices that take on such tasks. Needless to say that correct application of architecture patterns is crucial here.
Besides implementation, activities like testing also gives some extra challenges.

In addition, to make deployment easier, special platforms can be used to easily deliver and deploy Microservices. A good example of such a platform is Docker, that enables Containerization. We've planned some Wednesday Evening Trainings to take a dive in Docker in the upcoming months. Make sure you don't miss my blog posts on these sessions!

I'll be writing more on Microservices in a separate blog post I'll be writing shortly (promise!).



Back to this evenings Wednesday Evening Training...

In this session, Bart van Beek (one of our top Java developers) and I explained the basics of Microservices both from coding and architectural perspective. We've also taken a deep dive into code and did walkthrough of a real project (Java) with one of our clients. There was plenty of opportunity for Q&A and discussion.

We'll be definitely plan additional sessions on Microservices, since this architecture is here to stay and there is still much to learn and share to implement this architectural style effectively and efficiently.


Further reading

Do you want to read more on the topics in this post? Here are some links…

Microservices Resource Guide: https://www.martinfowler.com/microservices
Great articles and pattern descriptions of Martin Fowler: https://www.martinfowler.com/articles/enterprisePatterns.html
Microservice Design Patterns Catalog: http://microservicepatterns.org
Microservice Architecture Tutorial: https://www.tutorialspoint.com/microservice_architecture/index.htm
Designing, building, and operating microservices on Azure: https://docs.microsoft.com/en-us/azure/architecture/microservices
.NET Microservices: Architecture for Containerized .NET Applications: https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture
Microservices architecture style (challenges, best practices, etc.): https://docs.microsoft.com/en-us/azure/architecture/guide/architecture-styles/microservices

Past Wednesday Evening Trainings on all topics

You 'll find post of previous sessions here: https://www.linkedin.com/search/results/content/?keywords=%23wednesdayeveningtraining


#wednesdayeveningtraining #capgemini #lifeatcapgemini #microservices

12/1/18

Week #48's Wednesday Evening Training: a "klusavond" with a lot of math, workers and automata theory

In this week's Wednesday Evening Training: a "klusavond" with miscellaneous topics.

A "klusavond"?

Well, that's a typically Dutch word for a good old fashioned evening in which everybody works on his own projects and we have some demo's, presentations and discussions on various topics. Some weeks ago, during our Wednesday Evening Training evenings on Quantum computing and OpenGL, we decided that we definitely needed some refreshment of our basic math knowledge on vector calculus and complex numbers. Also, we also wanted to have a greater sense of the kind of problems that quantum computers can solve, that's why we wanted also to go deeper into the limits of the problem-solving capacity of classic computers. That required us to have some refreshment on automaton and problem theory as well. And what would be a Wednesday Evening Training evening be without programming? So we also added some explanation, a demo and some hands on labs on JavaScript workers.



On the math...

On the math, our Quantum computer champion Julian van Velzen and our OpenGL champion Quincy Jacobs had some nice exercises prepared for us. We worked out these assignments on the whiteboard together. Quincy even had programmed some nice online vector calculation simulations, which you can find in the resources below. Thanks Quincy, Julian and Bart van Beek, this is certainly useful for our workshops on OpenGL and Quantum computing!

The problem solving power of Automatons

Next, I gave some explanation on the various kinds of automatons and their ability to solve problems. Automatons, or abstract machines, abstract computers, are theoretical models of a computer hardware or software systems. They form the theoretical background of all modern computers. Using these models, we can say something about the type of problems that computers can solve, and how (efficiently) they can do that. There are multiple classes of automata, varying from primitive to powerful. The most commonly known are Combinational logic, Finite-state Machines, Pushdown automatons and Turing Machines. Turing Machines are the most powerful and they form the theoretical basis of our modern computers. They are able to solve multiple classes of (mathematical) problems in a more or less efficient way. And there are classes of problems that Turing Machine based are not able to solve at all. We did a walk through of these automaton classes and the problem classes that they can solve. Ultimately we discussed the problem class a classical Turing Machine based computer can never solve, but a Quantum Computer can.
More on this in my article on "Quantum computing: an introduction", I'll do an update shortly.


















JavaScript workers: dedicated workers, shared workers and service workers

Basically, a "worker" is a JavaScript program running in the background, that is: in a separate thread in your browser. That has the clear advantage that it does not affect the performance of your web page, but is limited, e.g. that t is not able to interact directly with your web page. Workers are, for example, used for running (asynchronous) "services" in the background of your web page, fetching resources, doing calculations, et cetera. In Progressive Web Apps (PWA), they form the backbone of an application. See my blogpost "This week's Wednesday Evening Training: Progressive Web Apps (PWA) and an intermezzo on cryptography" for more information on PWA's and some links to resources on PWA's. We discussed three types of web workers: dedicated workers, shared workers and service workers. In short: dedicated web workers (the most primitive type) only works for 1 web page, shared workers (a bit more advanced) can be used by multiple web pages (from the same origin) and service workers (the most advanced) can actually handle requests to your web page (from the origin). With service workers, you're able to create offline web sites (PWA) that can even support push messaging. By the way: browser support on shared workers does not look good (more on this), use Service workers instead.
This evening , we did a walk through of all types of workers above and some nice hands on labs on Service workers (see the links below).
We'll definitely continue with Service workers in our next Wednesday Evening Training session on Progressive Web Apps.

Further reading

Do you want to read more on the topics in this post? Here are some links…

On the math
QuincyJacobs/Vector-drawing-js: For explanation purposes during linear algebra: https://github.com/QuincyJacobs/Vector-drawing-js

On the Automaton theory
Quantum computing: an introduction (and a lot of links to resources): https://hansrontheweb.blogspot.com/2018/11/quantum-computing-introduction.html
Automata theory: https://en.wikipedia.org/wiki/Automata_theory
Turing machine: https://en.wikipedia.org/wiki/Turing_machine
Computational complexity theory: https://en.wikipedia.org/wiki/Computational_complexity_theory
Finally, a Problem That Only Quantum Computers Will Ever Be Able to Solve: https://www.quantamagazine.org/finally-a-problem-that-only-quantum-computers-will-ever-be-able-to-solve-20180621/

On Web Workers
Introduction, applications and code: https://html.spec.whatwg.org/multipage/workers.html
This week's Wednesday Evening Training: Progressive Web Apps (PWA) and an intermezzo on cryptography: https://hansrontheweb.blogspot.com/2018/10/this-weeks-wednesday-evening-training_18.html

Dedicated Workers:

Shared Workers:

Service Workers:


Past Wednesday Evening Trainings on all topics

You 'll find post of previous sessions here: https://www.linkedin.com/search/results/content/?keywords=%23wednesdayeveningtraining


#wednesdayeveningtraining #capgemini #lifeatcapgemini #html #javascript #quantumcomputing