Language for describing modbass-like protocols

Agents, Protocols and APIs
1

Worked a lot with various devices via serial interfaces (RS485, RS232). Quite often I was faced with the need to support one or another modbas-like protocol and each time I had to almost write a parser and message generator from scratch. The idea came to make a certain language for describing these protocols and configure an interpreter and generator based on a file written in this language. In order not to invent my own syntax, I chose yaml as a markup language.

version: 0.0.1
struct: [prefix, separator, address, separator, command, separator, data, separator, suffix]
parts:
  prefix:
    value: 1
  suffix:
    value: 2
  address:
    bytesCount : 4
  command :
    bytesCount : 2
  separator :
    value : 255
  data:

(Example protocol description 1)

version: 0.0.1
struct: [prefix, address, length, command, data, suffix]
parts:
  prefix:
    value: 1
  suffix:
    value: 2
  address:
    bytesCount : 4
  command :
    bytesCount : 2
  length :
    dependsOn : [command, data]
  data:

(Example protocol description 2)

For the sake of experimentation, I wrote a simple pre-prototype of an interpreter for this language in Kotlin. I designed it as a library and displayed the following interfaces.

 @Test
    fun exampleTest()
    {
        //read the protocol structure
        val filename = "TestData/ExampleLength.yaml"
        val text = File(filename).readText()
        val messageStruct = MessageStructFactory.fromYaml(text)

        //generate a message (sequence of bytes)
        val message : List<Int> = messageStruct.setAddress(listOf(13, 18)).setCommand(listOf(1)).setData(listOf(12,13,14,15,16,17)).build()
        println("Generated message : $message")// [1, 0, 0, 13, 18, 8, 0, 1, 12, 13, 14, 15, 16, 17, 2]

        //parse the message
        val parser = Parser(messageStruct.parts)
        val checkResult = parser.check(message)
        checkResult.errorList.forEach { println(it) }
        //check the message in case, for example, it was broken during transmission
        if(checkResult.success) {
            //if the message passed the check, extract the useful data
            val usefulData : UsefulData = parser.parse(message)
            println(usefulData)//UsefulData(address=[0, 0, 13, 18], command=[0, 1], data=[12, 13, 14, 15, 16, 17])
        }
        else
        {
            //do something else if it doesn’t work
            //return error
        }
    }

(Example using interpreter 1)

I am writing to understand how interesting this topic is within the framework of the openremote project. Is it worth developing it?

2 Likes
2

Hi,

Thanks for the message.

Definitely agree that a non java approach for configuring binary processing of messages from generic protocols is a useful thing to have; we have very limited string processing of messages but it is difficult to cater for all possibilities without ending up with a lot of options.

A scripting approach would provide more flexibility and is something on our roadmap.

In terms of processing data from protocols then at a basic level you just want to:

  1. Capture a message frame
  2. Convert data
  3. Update one or more attributes

Having some sort of data structure is just a middle step to get to the final step.

Using a DSL for doing steps 1&2 can work nicely until some conditional logic is needed (e.g. message frame size is conditional based on the message type).

YAML is definitely a simpler syntax but we make heavy use of JSON particularly for leveraging postgres JSON queries.

Let’s keep the discussion going as it is always good to get input from the community.

3

I suggested that there are different approaches to organizing message structure. I’m guessing there aren’t many of them. I will give examples.

4

“YAML is definitely a simpler syntax but we make heavy use of JSON particularly for leveraging postgres JSON queries.”
No problem, I can easily add JSON support.

5

“Using a DSL for doing steps 1&2 can work nicely until some conditional logic is needed (e.g. message frame size is conditional based on the message type).”
I have seen the protocols several times where Different messages have completely different structures.
The only thing that comes to mind in the early stages of project development is to force library users to use several DSL descriptions. The choice of which protocol structure to use rests with the library user.