Is Kubernetes the next big game changer for the IoT edge controller industry?
OSGi vs Kubernetes and micro-services design architecture
Before jumping to Kubernetes, I would like to make a quick parallel with another solution called OSGi, a service oriented architecture framework implemented in Java, that I have been using in the past and who is an interesting candidate for embedded controllers with limited resources. In particular, it provides the capability of deploying and updating, in live environment, bundled services taking advantage of the Java Classloader mechanisms to prevent service interruption.
So, the promise of having modular, self-containerized bundles accessible through a registry, being able to communicate with a publish subscribe mechanism to have a low coupled services interface is first, not something new in the context of embedded controllers and OSGi was still a choice I had in mind for our current stack development.
That said, I also felt constrained by the framework coding approach style and limited to Java while I wanted to take advantage of some modern AI/ML frameworks where Python is more dominant. Also as we are developing some of our own drivers in C/C++, using Java JNI wasn’t really appealing to me neither and my team not so familiar with Java.
Last but not least, I was also interested to utilize the possibility of having a grape of nodes in a cluster to perform distributed computation and load balancing. That configuration type brings me to work on a design we hope to get patented for an industrial application use case using Kubernetes as a core backend.
Kubernetes on the edge
As I wanted to keep similar advantages of what OSGi offers while solving the challenges mentioned above, I found Kubernetes and the micro-service architecture design pattern to be an approach that could give me a nice fit for developing our IoT middleware platform and I will explain more in details below why I think it is a serious game changer despite some current limitations.
Pro.
· Zero downtime strategy, the updates are applied in live environment with no reboot
· Fast pace release updates / security patches and new features when integrated with a CI / CD platform
· Flexible environment to build containerized services in different programming languages. NodeJS for the web, Python for ML/AI and C/C++ for drivers as an example
· Scalable and resilient application while deployed in multi-node cluster
· Accelerate the go-to-market for more added value services (especially AI/ML)
Cons.
· Log monitoring require special attention to be sure the system is not drifting
· Complex to install and fine tune, need high expertise but this can be facilitated with prebuilt systems who provide this “as a service” middleware like ours
· Resources are limited on edge controllers, so it is important to well optimize the image footprint, CPU and memory consumption for each micro-service
· Access to the physical devices (UART, I2C, etc.) through pods / containers requires specific settings with high privileges
DevOps methodology for IoT edge device and Kubernetes deployment rollout
I have been working with a wide variety of automation devices, global area controllers and SCADA systems and one thing I was always frustrated about is how slow it is as a developer to release and test new updates of my code.
Even today, the delivery process is quite inefficient because the toolchain is not adapted to a fast-paced release cycle.
It often goes through a lot of manual steps to build, test and deploy new code and also implies to reboot or flash the device to take the modifications into account.
One way to overcome this is to adopt the best practices from the DevOps world and reuse the same tools and workflow.
So choosing Kubernetes makes me quite happy about this, as it is possible to define declarative updates through Deployments definition and the orchestrator will manage the rollout of the new version of the micro-services once they are published to the registry through the CI pipeline.
It is also possible to rollback in case of failure or design more advanced strategy scenario with a canary or a blue/green release.
Why is it a game changer and for who?
· On the operational side
From my experience working as and with system integrator, one of the painful things was to release fixes and updates for controllers who were operating sensitive processes.
Planed updates are usually scheduled to minimize the impact on the customer side but that implies for the integrator to work on constrained time during weekends or late evening and if anything goes wrong it is uneasy to rollback or test quickly new changes / fixes.
So, what happen is usually few or no changes are applied, and the systems are replaced only when they break. Even critical security updates are not taken seriously today due to the complexity and impact on the customer process. This become a critical problem for those assets who need to be more and more connected to other systems used for data analytics.
Moreover, no added value is integrated after the initial commissioning of the installation and a retrofit means most of the time a full replacement of the current system which increase the capital expenditures.
Another point is the redundancy management, today even in data-centers there is a 50% fault tolerance on the monitoring devices for economical reason and practical deployment constraints. The issue is that it requires some specialized technician to intervene fast to respect the SLA (service level agreement) and therefore having more resilient infrastructure could reduce the operational cost.
· On the service provider side
Another important factor I have identified when I was working for energy management companies, is the difficulty / complexity to deploy added value services on the infrastructure and assets we were operating.
We were very constrained in the deployment cycle of our analytical models (energy consumption forecast for smart grid as an example) and once deployed on site it was very difficult to rollout new versions of our models that could be refined with the data collected on the processes we were monitoring.
Being able to continuously re-deploy fined tuned models and prevent them from drifting is necessary to provide the best optimizations on site and increase our operational margins.
Conclusion
The utilization of modern architecture like Kubernetes and micro-services design pattern offers on the edge, even on a single node, lot of value not only for the development process but also for the whole life cycle of the project, especially during the ongoing commissioning phase and act as true enabler to release seamlessly continuous updates efficiently and securely with no downtime on the process operation.
Even if there is still some considerations and limitations to look at as the arm64 based images availability, some strong partnership with ARM and Docker confirm the trend that the containerization technologies will be more and more dominant at the edge either.
Some challenges will remain for the GPU virtualization support and hardware acceleration for AI/ML frameworks like Tensorflow as Nvidia CUDA support for Docker arm64 version is still missing. But as the demand is strong and the arm community growing, it would be unlikely that Nvidia offer for its Jetson platform doesn’t evolve too.
The need for more intelligent and autonomous edge systems who integrate DNN (deep neural network) technologies will also help this adoption as it will facilitate the deployment of updated models that are retrained to reach the best possible fitting.
To conclude, I do believe that this industry segment will move in the coming years to embrace more agile methodology approach and take the benefit of this kind of architecture to facilitate the deployment, maintenance and operation of on-premise devices.
If you have any comments, experiences to share in this field I would be happy to hear from you!
Other articles
· Building Automation System security $>_ h@cking Data Centers like Mr. Robot (slides)
· Building Automation System security $>_ h@cking passwords (slides)
· Building Automation System security $>_ h@cking j0urney (slides)