Auto Prosioning Response with Service User credentials

Hatim · September 6, 2023, 3:13pm

Hi,

I am trying to create and asset using Auto Provisioning feature and it is working as expected.
To be honest, this is really great feature.

The problem is after the Auto Provisioning the client devices can’t connect with OpenRemote over MQTT because it does have any information about thew newly created service user and its secret.

How can I fetch the newly created user and its secret to connect mqtt to read/write attributes?
or Is there any way to send this information as part of AP request?

Regards,
Hatim

Rich · September 6, 2023, 9:30pm

Hi Hatim,

Glad to hear you got the auto provisioning working.

The idea is that you send the provisioning request and once you get a successful response your session is then authenticated as that service user; any time you lose connection you will have to go through the provisioning request again (but no additional asset will be created).

Hatim · September 6, 2023, 9:55pm

Hi Rich,

Ahh alright. This makes a lot of sense.
My mistake was, I was creating a new MQTT client once Auto Provisioning response is received.
I never thought that I already have a connected client authenticated using certificates so there wouldn’t’ be any need of creating new connection using user credentials - no wonder I am still a naive.

Thank you for your help here.

Regards,
Hatim

Hatim · September 7, 2023, 11:27am

Hi,

I tried to use the same MQTT client, unfortunately it does not seem to be work. I have not got any chance to troubleshoot. I could not find any error message in any logs. It’s just not getting message from OpenRemote to the client.

Rich · September 7, 2023, 11:40am

You subscribe to the provisioning response topic and wait for a success message to be received there?

From this success message you can extract the asset ID if needed for future attribute publishes.

Hatim · September 7, 2023, 12:03pm

Yes, I am getting response and getting the asset ID.
I am subscribing using that asset ID.
I can see the mqtt client status is connected.
Unfortunately, It can’t communicate. There must be some setting that I am missing here.

dcompent · December 23, 2023, 1:08am

@Rich I have run into effectively the same problem, but I think I have determined the reason/cause.

Either there is either a flaw with the automatic provisioning system, or I am severely missing something!

So I’m using an X.509 cert to spec, with a unique CN (Common Name) comprising a UUID, guaranteeing uniqueness. The provisioning handshake works as expected, with each derivative X.509 cert (with a unique UUID) being validated by the master X.509 cert in the “Auto provisioning” screen.
After the provisioning handshake, OpenRemote creates a ServiceUser, a copy of the Asset, and links the Asset to the ServiceUser, as expected.

But that’s where the good news comes to an end.

So in the X.509 authentication, the following all must be true:

The MQTT ClientID must match the X.509 CN (Common Name), which in this case is a UUID
The 2nd MQTT topic parameter must match the ClientID ( {realm}/{clientId}/writeattributevalue/{attributeName}/{assetId} ) as per User Guide: Manager APIs · openremote/openremote Wiki · GitHub
The 2nd MQTT topic parameter must ALSO match a ServiceUser in the corresponding Realm in order to be granted the necessary permissions to access the permitted Asset or Attribute. (“…authentication requires a ‘Service user’ username and secret”, [same link as in #2](User Guide: Manager APIs · openremote/openremote Wiki · GitHub, first paragraph)

Step #3 is where the problem is. Yes, OpenRemote generates a ServiceUser in the corresponding Realm, and assigns the newly created Asset to the ServiceUser…BUT! When generated, the ServiceUser’s name is prepended with “ps-” (https://github.com/openremote/openremote/blob/d07cc300538028eaab285100f8a3e9da0769dd92/manager/src/main/java/org/openremote/manager/mqtt/UserAssetProvisioningMQTTHandler.java#L312 with reference to https://github.com/openremote/openremote/blob/d07cc300538028eaab285100f8a3e9da0769dd92/manager/src/main/java/org/openremote/manager/mqtt/UserAssetProvisioningMQTTHandler.java#L107)

This means that step #3 results in the OpenRemote authenticator failing with “User: {realm}:{user} does not have permission=‘SEND’ on address {MQTT topic}”.
Result: it’s impossible for an X.509 automatically-provisioned field MQTT device to be able to actually access its OpenRemote Asset!
(If the Assets are manually configured per each device, then this is not a problem. But “manual != automatic.”)

I banged my head against the wall for several hours trying to figure this out, as the auto-provisioning MQTT handshake with the X.509 was working correctly–but no matter what, any attempt to write to the corresponding Asset Attribute would result in OpenRemote terminating the MQTT connection and popping the above error into the logs.
Finally, I manually created a ServiceUser (as the auto-generated ServiceUser name field is disabled and cannot be edited) with the direct UUID (i.e. the X.509 CN). Then I set the appropriate permissions (well, in frustration, this was “full write”, same as on every other ServiceUser and Asset)…and voila, if the “writeattributevalue” Publish didn’t go right through for the first time in forever, and OpenRemote didn’t kick the MQTT client off.

Interestingly (and fortunately!) the auto-generated Secret for the ServiceUser does not seem to matter: the MQTT connection username/password used are the ones for the auto-provisioning ServiceUser (as we do have to go through the X.509 authentication sequence anyway).

Am I missing something here?

dcompent · December 23, 2023, 5:37pm

github.com

openremote/openremote/blob/d07cc300538028eaab285100f8a3e9da0769dd92/manager/src/main/java/org/openremote/manager/mqtt/UserAssetProvisioningMQTTHandler.java#L367-L370


      
          // Authenticate the connection using this service user's credentials - this will also update the connection's subject
          connection.setSubject(null); // Clear existing anonymous subject
          mqttBrokerService.securityManager.authenticate(realm + ":" + serviceUser.getUsername(), serviceUser.getSecret(), connection, null);
          mqttBrokerService.notifyConnectionAuthenticated(connection);

@Hatim Here is where the connection is automatically switched from the authentication user to the created/associated ServiceUser–with the corresponding username and password. As a result, clients connecting via X.509 cert do not need to know the ServiceUser password; technically they don’t need to know the ServiceUser username either–except in what I seem to be running into above, where the topic permissions block any writes if the ClientID does not match the ServiceUser username.

Rich · February 27, 2024, 6:24pm

Hey @dcompent sorry for the long delay in getting back to you, this got lost in my mailbox.

As you mention you shouldn’t need to know about the service user on your MQTT client as the backend does the authentication with keycloak based on the X.509 certificate provided.

I don’t know if using a UUID for the device identifier is causing problems in this process somehow.

The auto provisioning is used a lot in our customer projects but the device identifiers we use are simpler strings like device0001

dcompent · February 28, 2024, 2:23pm

I don’t see how it could; rather, what I’m seeing is the code specifically prepending a “ps-” onto the given device identifier, as I linked above and again here: https://github.com/openremote/openremote/blob/d07cc300538028eaab285100f8a3e9da0769dd92/manager/src/main/java/org/openremote/manager/mqtt/UserAssetProvisioningMQTTHandler.java#L312

Also like I mentioned above, the automatic device provisioning is working just fine. Where it fails is in the MQTT Write function with restricted permissions–because the device identifier does not match the internal service worker identifier, and that is the result of the latter having been prepended so as not to match the device identifier.