Knolar Documentation

# CPH Tool

# Knolar Gaterway

CPH Tool is independent software, which has been designed to complement CPH in its communication with data sources. This component has two usage modes:

  • MQTT accumulator: In this mode CPH Tool works as a buffer. Events are received and these are grouped into event packages. This mode is recommended when the source has a high rate of publications per second, as the events arrive CPH MQTT brokew with lower cadence and grouped in packages.
  • Bidirectional Gateway OPC-UA -> MQTT: In this mode the CPH Tool works as a Gateway (Translator) of the industrial protocol OPC-UA to MQTT. The configuration also allows us to do a direct translation or to add a buffer to the events in order to group them.
    • CPH->OPC-UA_: If activated, the Gateway also allows writing to the configured OPC-UA server (writing is done through POST requests).

# Mode: Bidirectional Gateway OPC-UA => MQTT/SSL

Gaterway1

The gateway is a software that translates the OPC industrial protocol in its OPC-UA version to MQTT with SSL and injects it into a CPH broker.

For ease of use, it has been encapsulated in a private Docker container. You need to login with your Docker Hub username to download it. You can signup in the Docker Hub website and then send the username by email to: admin@knolar.io.

# OPC-UA Gateway to MQTT: Installation and deployment instructions

The following steps must be followed:

  • Create a Real Time / Near Real Time ingest on the web platform to obtain the necessary endpoint and certificates, as well as to prepare the platform.
  • Download the certificates and copy them into a certs folder.
  • Create a configuration file (see below) with the ingest data and copy it to a cfg folder.

These folders must be at the same level, with a structure similar to the following:

.
├── certs
│   ├── cert-<ALIAS_INGESTA>-cert.pem.crt
│   ├── cert-<ALIAS_INGESTA>-privateKey.pem.key
│   └── cert-<ALIAS_INGESTA>-publicKey.pem.key
└── cfg
   ├── config.template.yaml                        # By default
   └── config.yaml
  • Launch the gateway. Note: The command requires absolute paths to access files outside the container. See example below
$> docker login (Required as it is a private image)

# Linux & MacOs X
$> docker run -it --rm -v $(pwd)/certs/:/opt/certs/ -v $(pwd)/cfg/:/opt/cfg/ -e CFG_PATH=/opt/cfg/config.yaml batiscafo/gateway:latest

# Windows
$> docker run -it --rm -v c:/project/certs/:/opt/certs/ -v c:/project/cfg/:/opt/cfg/ -e CFG_PATH=/opt/cfg/config.yaml batiscafo/gateway:latest

# Configuration file

The configuration file uses the yaml standard and defines the OPC UA subscription endpoints from which to obtain data and the MQTT endpoint to send them to. It can also define if data should be enriched or buffered before sending. See example below:

# Config Sample
api:
    port: 3000

# Optional. If this group is included "<site><source><separator>" will be added to the beginning of "fieldId" field
context:
    site: '02'
    source: 'OFD'
    separator: '.'

opcua_broker:
    # Flag to allow to send requests to the OPC UA server
    api_writable: true
    endpoint: opc.tcp://opcuaserver.com:48010
    # Self-generated field with the name of the server to which we subscribe
    fieldId: CustomTag
    keys:
        - ns=3;s=AirConditioner_1.Temperature
        - ns=3;s=AirConditioner_1.State
        - ns=3;s=AirConditioner_1.PowerConsumption
        - ns=3;s=AirConditioner_1.Humidity
        - ns=3;s=AirConditioner_2.Temperature
        - ns=3;s=AirConditioner_2.State
        - ns=3;s=AirConditioner_2.PowerConsumption
        - ns=3;s=AirConditioner_2.Humidity
        - ns=3;s=AirConditioner_3.Temperature
        - ns=3;s=AirConditioner_3.State
        - ns=3;s=AirConditioner_3.PowerConsumption
        - ns=3;s=AirConditioner_3.Humidity
        - ns=3;s=AirConditioner_4.Temperature
        - ns=3;s=AirConditioner_4.State
        - ns=3;s=AirConditioner_4.PowerConsumption
        - ns=3;s=AirConditioner_4.Humidity
        - ns=3;s=AirConditioner_5.Temperature
        - ns=3;s=AirConditioner_5.State
        - ns=3;s=AirConditioner_5.PowerConsumption
        - ns=3;s=AirConditioner_6.Humidity
        - ns=3;s=AirConditioner_7.Temperature
        - ns=3;s=AirConditioner_7.State
        - ns=3;s=AirConditioner_7.PowerConsumption
        - ns=3;s=AirConditioner_7.Humidity
        - ns=3;s=AirConditioner_8.Temperature
        - ns=3;s=AirConditioner_8.State
        - ns=3;s=AirConditioner_8.PowerConsumption
        - ns=3;s=AirConditioner_8.Humidity
        - ns=3;s=AirConditioner_8.Temperature
        - ns=3;s=AirConditioner_9.State
        - ns=3;s=AirConditioner_9.PowerConsumption
        - ns=3;s=AirConditioner_9.Humidity
        - ns=3;s=AirConditioner_10.Temperature
        - ns=3;s=AirConditioner_10.State
        - ns=3;s=AirConditioner_10.PowerConsumption
        - ns=3;s=AirConditioner_10.Humidity

mqtt_gateway:
    method:
        # Required include one of the two modes

        # Direct sent
        #mode: direct

        # Buffer with the following settings:
        mode: buffer
        time: 10 # Seconds
        volume: 5 # Kbs


    # Real Time Gateway / Broker Configuration
    channel: real_time #real_time o near_real_time
    region: eu-west-1
    topic: <TOPIC>
    endpoint: <ENDPOINT>
    port: 8883
    auth:
        ca: 'base/AmazonRootCA1.pem'
        cert: 'certs/cert-<ALIAS_INGESTA>-cert.pem.crt'
        key: 'certs/cert-<ALIAS_INGESTA>-privateKey.pem.key'

You can make a copy of the template file, rename it to config.yaml and move it to cfg before starting the container.

# CPH Data Platform -> OPC-UA

Gaterway2

First of all you must check that you have write permissions on the OPC-UA server.

To activate this functionality, in the config.yaml configuration file, you must set to "true" the field "api_writable" of the group "opcua_broker". From CPH you can write to an OPC-UA server using the following tools:

  • Jupyter Hub (Example in the welcome notebook)
  • Rapidminer

# Limitations

There is a limitation on the size of the messages that should be considered:

  • mode: pipe: The maximum message size accepted by CPH platform is 128KB.
  • mode: buffer: The maximum message size accepted by CPH platform is 128KB, i.e., volume: (in config.yaml) cannot exceed 100 (safety margin).

# MQTT Accumulator

CPH Tool is an independent software, designed to support CPH on its communication with the datasources. This component has two use modes:

  • MQTT Accumulator: In this mode, CPH Tool works as a buffer, where events are received and grouped in event packages. This mode is advised when the origin has a high publications per second rate, so that the CPH's MQTT broker receives event packages with a lower rate of publications and grouped.
  • Bidirectional Gateway OPC-UA -> MQTT: In this mode, CPH Tool works as a Gateway (translator) from the industrial protocol OPC-UA to MQTT. It is possible to configure either a direct translation or to add an event buffer that allows for grouping.
    • CPH->OPC-UA_: When activated, the Gateway allows to write in the configured OPC-UA server as well. This writing is performed with POST requests.

# Mode: MQTT Accumulator

Gaterway3

The MQTT Accumulator is a piece of software that accumulates MQTT messages and injects them in a Broker of the CPH product. The accumulator's goal is to act as a buffer between the publishing devices and the CPH broker so that the broker receives message packages instead of single messages. It is advised for high rates of publications per second.

For ease of use, it has been encapsulated in a private Docker container. You need to login with your Docker Hub username to download it. You can signup in the Docker Hub website and then send the username by email to: admin@knolar.io.

# MQTT Accumulator: Installation and deployment instructions

The following steps must be followed:

  • Create a Real Time / Near Real Time ingest on the web platform to obtain the necessary endpoint and certificates, as well as to prepare the platform.
  • Download the certificates and copy them into a certs folder.
  • Create a configuration file (see below) with the ingest data and copy it in a "cfg" folder. These folders must be at the same level, with a similar structure to the following:
.
├── certs
│   ├── cert-<ALIAS_INGESTA>-cert.pem.crt
│   ├── cert-<ALIAS_INGESTA>-privateKey.pem.key
│   └── cert-<ALIAS_INGESTA>-publicKey.pem.key
└── cfg
   ├── config.template.yaml                        # Set by default
   └── config.yaml
  • Launch the gateway. Note: The command requires absolute paths for access to files that are external to the container. See the example below:
$> docker login (Compulsory as we are downloading a private image)

# Linux & MacOs X
$> docker run -it --rm -v $(pwd)/certs/:/opt/certs/ -v $(pwd)/cfg/:/opt/cfg/ -e CFG_PATH=/opt/cfg/config.yaml batiscafo/gateway:latest

# Windows
$> docker run -it --rm -v c:/project/certs/:/opt/certs/ -v c:/project/cfg/:/opt/cfg/ -e CFG_PATH=/opt/cfg/config.yaml batiscafo/gateway:latest

# Configuration file

The configuration file uses the yaml standard and defines the MQTT inbound port, topic, and endpoint, as well as whether the messages must be complemented or a buffer must be set up in order to dampen the publications per second rate:

# Config Sample
api:
    port: 3000

mqtt_broker:
    port: 8883
    topic: <TOPIC>

mqtt_gateway:
    method:
        # It is compulsory to state one of the two below

        # Direct delivery
        #mode: direct

        # Buffer with the following conditions:
        mode: buffer
        time: 10 # Seconds
        volume: 5 # Kbs


    channel: real_time #real time o near_real_time
    region: eu-west-1
    topic: <TOPIC>
    endpoint: <ENDPOINT>
    port: 8883
    auth:
        ca: 'base/AmazonRootCA1.pem'
        cert: 'certs/cert-<ALIAS_INGESTA>-cert.pem.crt'
        key: 'certs/cert-<ALIAS_INGESTA>-privateKey.pem.key'

You can make a copy of the template file, rename it to config.yaml and move it to cfg before launching the container.

# Limits

There is a limit in the message size that must be taken into account:

  • mode: direct: The maximum size of a message that the CPH platform accepts is 128KB.
  • mode: buffer: The maximum size of a message that the CPH platform accepts is 128KB, namely, volume: (in config.yaml) cannot exceed the value of 100 (safety margin).

← Machine Learning →