Fleet Management/telematics integration updated to latest

panos · April 12, 2024, 7:51am

If the two containers are living in the same machine, you can use docker container inspect on the manager container to see what IP is being used. You can then use <container IP>:1883 to use unencrypted MQTT. This should circumvent the reverse proxy and reach the MQTT broker of OpenRemote directly. If you don’t see anything being logged in the manager, it means that the requests do not make it to where they should. If you want to look at even more granular logs in OpenRemote, you can specify your own logging.properties file. Here’s an example, look at lines 40 to 50: openremote/manager/src/main/resources/logging-dev.properties at master · openremote/openremote · GitHub

Information about how to provide your own logging file:

glodea · April 13, 2024, 12:47pm

I have performed all the checks but hitting still against the same wall, until I have discovered these entries in the Manager UI log:

04/12/2024 21:19:19	INFO	API	ActiveMQORSecurityManager	Un-supported request pub: topic=master/fmc003rd/86463xxxxx189/data, connection=/172.18.0.5:42784, clientID=86463xxxxx189, subject=anonymous
04/12/2024 21:19:18	INFO	API	ActiveMQORSecurityManager	Un-supported request sub: topic=master/fmc003rd/86463xxxxx189/commands, connection=/172.18.0.5:42784, clientID=86463xxxxx189, subject=anonymous

I thought that the issue might be that, while a teltonika device directly communicating to the MQTT broker, logs in using a certificate, the TCP server is logging in as anonymous and this looks like to be not allowed.
Looking at the MQTT API specifications, this seems to be the case too:

MQTT API (MQTT Broker)

Another publish subscribe API, authentication requires a ‘Service user’ username and secret and is done using standard MQTT username and password mechanism, to connect to the broker:

Host: Host of the manager (e.g. demo.openremote.io)
Port: 8883 (if running with SSL i.e. the standard stack with reverse SSL proxy) 1883 (if running without SSL proxy)
Encryption/TLS: true (port 8883) false (port 1883)
Username: {realm}:{username}
Password: {secret}
ClientId: anything you like (but don’t use the same ClientId more than once)

I have hence modified the index.ts inserting following:
var opts = {
orOpts: {
realm: process.env.orOpts__realm,
teltonika_keyword: process.env.orOpts__teltonika_keyword,
dataTopic: process.env.orOpts__dataTopic,
commandTopic: process.env.orOpts__commandTopic,
},
mqttOptions: {
host: process.env.mqttOptions__host,
port: parseInt(process.env.mqttOptions__port),
username: process.env.mqttOptions__username, // username
password: process.env.mqttOptions__password // password
},
udp_options: {
port: parseInt(process.env.udpServerOptions__port)
}
};

and added following to the .env
mqttOptions__username=master:serviceuser
mqttOptions__password=

Now I see it is subscribing:

04/13/2024 14:23:34	INFO	API	UserAssetProvisioningMQTTHandler	Adding publish consumer for topic ‘provisioning/+/request’: handler=UserAssetProvisioningMQTTHandler
04/13/2024 14:23:34	INFO	API	DefaultMQTTHandler	Adding publish consumer for topic ‘+/+/writeattributevalue/#’: handler=DefaultMQTTHandler

But I don’t manage to get the device visible…still struggling with that.
I need to understand the mechanism you use to identify which json file to use, depending on the device model and how this is created automatically in the assets…

At least a step forward.

panos · April 13, 2024, 2:42pm

Hi,

Couple of things here;

Due to limitations by Teltonika, users are not allowed to set their own MQTT connection credentials, meaning that you cannot use username and password to log in to an MQTT broker (with current Teltonika firmware).

Authentication with the device currently does not work using the certificates you upload. The certificates you upload are only there to allow secure communication with the broker (MQTTs, TLS). To be clear, the certificates are easily publically accessible, they’re what your browser automatically uses to connect to, for instance, the Manager UI.

To circumvent this, I have tweaked the Teltonika MQTT message handler to not require authentication (something required as you read in the MQTT API documentation). This can be seen as a message in your OpenRemote logs on startup:

Anonymous MQTT connections are allowed, only for the Teltonika Telematics devices, until auto-provisioning is fully implemented or until Teltonika Telematics devices allow user-defined username and password MQTT login.

Now to the interesting stuff;

I see that you edited the code in the TCP server to remove the /teltonika/ part. This means that, instead of your MQTT topic looking like master/fmc003rd/teltonika/86463xxxxx189/data, it looks like master/fmc003rd/86463xxxxx189/data.

OpenRemote uses different handlers that handle connections/messages for different services. For Teltonika devices, we use {realm}/{client ID}/teltonika/... . Now the way this works is, if a topic to which a message is sent does start with {realm}/{client ID}/teltonika/, then the Teltonika message handler will deal with the message. Create/update an asset, etc. If the topic does not match with any handler, it will print the Un-supported request message you see.

This confirms that the TCP server does correctly connect to the MQTT broker. Thus, this changes the problem completely, since we can assume that the TCP server does function correctly, but something between OpenRemote itself and the Teltonika message handler is going wrong.

I will investigate the issue.

panos · April 13, 2024, 4:03pm

I published a new version for the TCP server, that now logs if the server is connected to the broker or not, and also uses the protocol environment variable to either use or not use TLS encryption for the connection. mqtt is for unencrypted, mqtts is for unencrypted. Let me know if this helps.

panos · April 13, 2024, 4:50pm

If you want to view more logs from OpenRemote’s proxy, you can update the environment variable to PROXY_LOGLEVEL=info, this logs all connections to your proxy

glodea · April 15, 2024, 4:01pm

I have scratched the TCP Server and pulled this new version. Great! It works:

The assets you see in the picture are communicating through TCP and not MQTT. It works!

Thanks!

Still one question: I have created different new Teltonika Model Configuration Assets and pasted the JSON files which I have generated following your guide, using Junyper Notebook. I am not sure if I did it right, meaning: how is the mechanism of a new asset being logged in, to choose the right model?

glodea · April 16, 2024, 10:21am

Hi @panos,

just a heads up: I am currently testing the solution intensively, since it’s working.
It’s stopping with some errors periodically and I am trying to understand what the issue is. I guess some error handling might be needed.
I get this from the log:
/usr/src/app/out/index.js:38
if (!clients[imei].client.connected) {
^

TypeError: Cannot read property ‘client’ of undefined
at UdpServerManager. (/usr/src/app/out/index.js:38:24)
at UdpServerManager.emit (events.js:400:28)
at UdpServerManager.sendMqttMessage (/usr/src/app/out/UdpServerManager.js:36:14)
at /usr/src/app/out/UdpServerManager.js:79:39
at Array.forEach ()
at Socket. (/usr/src/app/out/UdpServerManager.js:78:146)
at Socket.emit (events.js:400:28)
at addChunk (internal/streams/readable.js:293:12)
at readableAddChunk (internal/streams/readable.js:267:9)
at Socket.Readable.push (internal/streams/readable.js:206:10)
[1]+ Done nohup docker logs edxxxxxxc4 -f > logfile.log 2>&1

Addendum: I have implemented a series of error avoidance and tracing measures. I am testing them right now. Once working and stable I can share with you.

glodea · April 18, 2024, 5:08pm

@panos update:
I have changed:
if (!clients[imei]) {

with:
if (!clients[imei] || !clients[imei].client.connected) {

and it is running since 36h without any errors.

I now need to address the fact that Beacon is passed as integer instead of hex, as it happens if I use direct MQTT.

Cheers,
Antonio

glodea · May 16, 2024, 3:30pm

Hi @panos,
I have a doubt regarding the setup for devices different from FMC003.
I have generated a new Teltonika Device Configuration, generated the json file as per your instructions, copied and pasted the contents in the Teltonika Parameter Data. By clicking on the arrow it has generated the Teltonika parameter map:

When I connect the device it does not show any additional parameters other then the GPS related ones. I am missing beacons, Eye Sensors, CAN Bus data etc. I guess I have done something wrong.
Is it not enough to insert the Model number to map it to the right configuration like this?

Any suggestions about potential misunderstanding from my side? I could read the full source code, but a quick hint might save me a lot of time.
Thanks
Antonio

panos · May 16, 2024, 5:39pm

Hi Antonio,

I’m gonna need some more information to help you out with this. There are many reasons why this could happen, and I think that this is starting to become Teltonika’s fault. Can you please send me the JSON file you pasted into the parameter data field? Maybe there is an issue there. Also, please check the logs/container logs and let me know if there is an issue there. Sometimes the JSON output from the parameter ID parser doesn’t really work because Teltonika’s wiki tables are not always consistent. Also, I am assuming that the data being sent from the device does contain those values. If you can, also attach a dummy payload from the device so that I can start testing stuff out (dont forget to remove any sensitive info).

Best regards,

Panos

panos · May 16, 2024, 5:51pm

@glodea forgot to ping you