It was a summer day, I was alone in the Electroplating department, hacking a FlexKraft plating rectifier from Kraft Powercon. I was trying to read the ModBus communication between the rectifier and the HID (Human Interface Device). It’s an environment with large chemical baths, pumps and pipes everywhere and that industrial white noise. As I began to understand the data flowing between systems, I looked around and remembered the Batman movie where HarveyDent got half his face burned off. The chemicals in this area are not that dangerous, but they defined the name of the project: HarveyDent.
The electroplating department, like many industrial environments, has a large number of electromechanical systems. Some with simple interfaces, such as a thermo-controller, others more complex, where control is via a touch screen. In one way or another, all these systems have something in common. It is not possible to collect data in real time!
Adding sensors that can provide some kind of real-time data could be a possibility. However, it would not provide the same information presented by the system, due to several factors, and would lead to questions and uncertainties. Using the data generated by the equipment that controls the operation will eliminate doubts that may arise later.
My solution to gathering data from the existing systems was to implement a MITM (Man in the Middle) attack or capture raw network traffic. One of the systems communicates using the ModBus protocol, it’s a client/server data communication protocol published by Modicon in 1979. It’s widely used for industrial applications and is easy to implement. I also learned that it is an insecure protocol, as I was able to collect the data using an RS485 to USB adapter connected to an embedded computer. Since I was able to read the raw data, it was only a matter of time to decode the bytes and find out which ones were related to the measurement I was looking for. I use Go to create the code to decode the information read from the devices.
The initial idea was to collect a lot of data points, put them into a machine learning algorithm and get a result that would help us understand why sometimes the same product, under the same conditions, has different results.
From the moment I managed to get the measurements flowing, I realized that it was necessary to implement something well designed to collect data automatically. This would also allow traceability of the equipment and tools used to produce the final product.
System architeture
The embedded computers collect the data in real time, in many cases 10 measurements per second, which is too much. Considering the speed and also the asynchronous way that many sensors provide. I decided to use redisDB running in a Docker container to collect the data. I wrote another program in Go that also allows flexibility to define the amount of data collected. The data is read from redisDB and written to InfluxDB, a widely used timeseries database.
User Interface
A key element was to provide a friendly experience for the operators to make product selections and record final measurements. To this end, I created a complete web application using the cakePHP framework for rapid development of the PHP code. In this part of the system, general information such as the type of product and the type of bath to be produced is defined. The system generates a serial number for each product and uses it to link all the measurements stored in influxDB.
Traceability
One of the innovations that I am very proud to have implemented was the use of a graph database to store product data. For this application I chose neo4j. The advantage of such a database is the use of connected nodes. A bath is a node connected to a product type, which is connected to a frame that has been used in many baths with others product types. With this information available, we can identify the relationship between a final product and the entire production chain.
Lessons learned
Due to the nature of this production line, the project was built completely off-grid and offline. There was no AWS available to create virtual machines. Everything from server, network, electrical connections, from hardware to software, everything was done by myself. I tested several solutions, some worked well, others failed. I’ve learned a lot about how to make IIoT work from the ground up, from data collection to data consumption. This project was the main motivation for writing Eduardo’s Digitalization Manifesto.
This project was the main motivation for writing Eduardo’s Digitalisation Manifesto.