Issues with mapped network ports, mapped volumes, map tiles and SmartCity demo

Hello to all!

I tried to deploy Openremote 3.0 using the pre-built docker images on a Raspi4, after reading all the respective configuration/deployment information.

After a lot of trial and error, I managed to setup the platform, however I’d like to point out the following issues:

(1) Volume mapping: As with other docker containers/services, I prefer to assign volume mappings to local storage than using docker volumes. I have created a dummy user (let’s call it “dockeruser”) which has uid/gid of 1010. Therefore, I added the following to all environment sections within deploy.xml:

PUID: 1010
PGID: 1010

Needless to say, these are not respected during deployment. For example, postgresql creates the postgresql data folder with ownership systemd-coredump:dockeruser. The systemd-coredump user has uid & gid of 999. On another note, the proxy container creates all files within the proxy-data mapped volume using root:root as ownership (not dockeruser:dockeruser, that is). I could not check the deployment data volume as the deployment container when executed does not populate the volume. Instead, in order for docker deployment to go forward, I manually copied all contents of the “deployment” folder on git to the deployment data volume.

What are the correct configuration so that openremote containers create volumes respecting the local user (i.e. “dockeruser”) with id 1010?

(2) I am running another container on the docker platform that makes use of the 443 TCP port. Thus, I tried to create the proxy container using the following port assignments:

ports:

  • “11080:80”
  • “11443:443”
  • “18883:8883”

Of course, I also modified the public hostname, issued and assigned an SSL certificate and modified the “manager” section of deploy.yml file accordingly. After I log into the manager, the map tries to load but never succeeds. I investigated the issue using the browser’s dev console and I found out that the map tries to load using a non-valid URL.

In detail, after logging into the https://myserver.com:11443/manager/, I get a blank screen instead of a loaded map.

The browser’s console shows: “Uncaught TypeError: Request constructor: https://myserver.com:11443:443/shared/map/sprites/bright-v9.json is not a valid URL.”

Clearly, the code adds “:443” to the URL and does not respect the IDENTITY_NETWORK_HOST (srv-docker-1.sunrise.brainwash.gr) & IDENTITY_NETWORK_WEBSERVER_PORT (11443) docker deployment values!

(3) I tried to change Realm and check out the smartcity one, but I get a reply that the smartcity realm cannot be located. How can this be activated? Within deploy.yml (manager block), I have already opted to install the demo information:

SETUP_IMPORT_DEMO_USERS: ‘true’
SETUP_IMPORT_DEMO_ASSETS: ‘true’
SETUP_IMPORT_DEMO_RULES: ‘true’
SETUP_IMPORT_DEMO_SCENES: ‘true’
SETUP_IMPORT_DEMO_AGENT: ‘true’
SETUP_IMPORT_DEMO_AGENT_KNX: ‘false’
SETUP_IMPORT_DEMO_AGENT_VELBUS: ‘false’

Docker images information (architecture: arm64)
proxy:latest, created 2021-03-05 22:05:55, ID sha256:6832433ec8a4e4cd3a469dbb45bd74dc134664deb775dbf084d922ec703bf856
manager:latest, created 2021-03-09 12:12:47, ID sha256:1bfaebc7a7dd802fd220a6bc622eda7d76905dcc77f66ee3f38edb8cf463f590
postgresql:latest, created 2021-01-31 12:45:13, ID sha256:4c3ebb041b7a460e74ae7c5ae7a37bf628eb0bb16044cdec0fac09c7ffbf9749
deployment:latest, created 2021-03-08 13:19:36, ID sha256:c4290e863da598f3422d06d3d430b777fd145a13d3bc5bc8dcc662d842e6df1f
keycloak:latest, created 2021-03-06 10:22:18, ID sha256:82aa9244858c1929f42c64c5ff1a9d86d485c2657968547d8b81deb83edc0e50

Hi,

at the moment we are busy to release a new deployment mechanism for our stack therefore :latest images can be a little bit unstable and using them can result in strange issues. There should be a press release within few days when we are done with this update where you should find the preferred and supported way to deploy the openremote stack. For a short note, we haven’t tested the stuff you mentioned, i.e. local storage for volume mapping and different UID and GIDs. Some images, like postgresql and keycloak are using stock base images to build, therefore they can have different users set in their Dockerfile.

Hi Michal,

Thank you for your prompt reply. I am following the development branch, however I thought that some basic areas (volume mapping or port mapping for example) were completed.

Of course, building the images locally can help adjust several parameters, however when customisations start the connection to the development tree starts to deviate :slight_smile:

If you need any help with testing the deployment as it progresses, I could lend out a hand if required!

There are still a lot of moving parts. For last few days we haven’t even had a working keycloak image for arm64, as the stock image from RedHat is Intel only. We’ve decided to create it for arm64 because of Apple M1 silicone release and you have it for RP4 as a side effect, although not fully tested yet. There are so many different configurations possible and it proves to be hard to get it right for each one. In the spirit of unifying the deployment experience I’ve started a CLI tool, which should work better than trying to follow frequently outdated README’s. If you are willing to help in testing and improve it go to GitHub - openremote/openremote: OpenRemote v3

In case you want a specific scenario to be implemented then let me know and we can try to implement it, or even better, as the cli is opensource you can add your scenario to it and create a pull request :slight_smile:

at this moment you can deploy it on your RP with:

openremote-cli deploy --dnsname srv-docker-1.sunrise.brainwash.gr -v

With the -v switch you see which commands are executed and you can simply copy them from there instead from README. This should be less error prone and more stable as I’ve frozen a different tag for images instead of bleeding edge latest one :wink:

Will follow your suggestion! CLI here I come :slight_smile:

Last but not least, I followed the instructions at How to work with MQTT · openremote/openremote Wiki · GitHub to publish some data to an asset I created within manager. I managed to connect, however when I try to publish or subscribe to a topic, I am not allowed to. I get:

manager_1 | 2021-03-09 17:00:31.708 WARNING [Messaging-AssetQueue-11 ] ote.manager.asset.AssetProcessingService : Error processing from CLIENT - INSUFFICIENT_ACCESS: AttributeEvent{timestamp=Tue Mar 09 17:00:31 EET 2021, attributeState=AttributeState{ref=AttributeRef{id=‘7TxncYfRyTXeoxI1QAblGE’, name=‘temperature’}, value=25, deleted=false}}

As you might suppose, 7TxncYfRyTXeoxI1QAblGE is the asset ID, I got it from from the URL within the manager (https://myserver.com/manager/#!assets/false/7TxncYfRyTXeoxI1QAblGE) when editing the asset as I could not locate it somewhere. I have given full write rights (as the admin user) to the service-account-d0909253-2edd-46bd-ad02-cd037163ac1d user that emerged within the manager, as soon as the client with ID d0909253-2edd-46bd-ad02-cd037163ac1d was created in the master realm within keycloak.

How can I give access to this clientID so that I can post values and publish/subscribe for this asset or for any asset?

Thank you!

MQTT is yet another bleeding edge feature, which is coded as we speak. Maybe the developer @Michael can help with this question?

Hi mx5gr,

Could you maybe share the ClientId and username used for the MQTT connection?

It’s important that CliendId starts with the realm prefix, for e.g. master_randomId and the username should be the client id as found in Keycloak. See attachment.

Kind Regards,
Michael