Breaking Changes to latest docker images

About OpenRemote
1

This topic is here for us to communicate any significant changes to the code base that will appear in the latest nightly docker images with the intention of helping users update their custom projects.

3

A breaking change has been made to the MQTT topic structure for asset and attribute publish and subscribe, please see the wiki page:

4

The latest manager docker image has breaking DB schema and will required a clean install.

5

Latest keycloak docker image is now using v18 which has significant changes to what we used previously. Environment variables in docker compose files have been renamed accordingly so be aware if using your own compose files.

6

I’ve simply copied keycloak section from staging.demo and now in custom project it does not redirct to localhost after login. However, this section doesn’t look right to me. It does:

    volumes:
      - ./deployment:/deployment

which results in empty folder. Anyway, elaboration on significant changes and renamed accordingly would be helpful.

7

Indeed that volume mapping is incorrect but there are no custom keycloak themes in the main repo so it is not relevant but I will update this mapping.

8

Note that we have changed the Insights page, adding the option to create multiple dashboards. If you have updated to the latest OpenRemote Docker image you might get an error message when navigating to the Insights page in the manager.
To resolve this, login directly in the Keycloak Admin Console (e.g. localhost:8080/auth) and:

  • Select your realm(s)
  • Go to ‘Clients’ and select ‘openremote’
  • Select ‘Roles’ (at the top)
  • Add the roles ‘read:insights’ and ‘write:insights’

When you now login into the manager, in the respective realm(s), you can toggle these roles per user (also for yourself as ‘admin’). From here onwards, users with these roles enabled will see the new Insights page. Enjoy trying it out!

1 Like
10
11
12

Latest docker image has the following breaking changes note in particular the need to update groovy rules (most of these only affect java code extensions):

  • Any groovy rules that directly used AssetState should be replaced with org.openremote.model.attribute.AttributeInfo (imports or inline usages)
  • Serialised AttributeEvent.assetState property has now been removed and AttributeEvent.ref and AttributeEvent.value should be used instead (deserialisation of AttributeEvents using the old assetState field in JSON are still supported for now TBR)
  • AttributeEvent.getAttributeRef() renamed AttributeEvent.getRef()
    • AttributeEvent.getAttributeState() renamed AttributeEvent.getState()
  • AttributeRef.getAttributeName()renamedAttributeRef.getName()
  • Protocol.updateLinkedAttribute() methods - AttributeState now removed, use AttributeRef and value instead
  • AbstractProtocol.doLinkedAttributeWrite(Attribute<?> attribute, U agentLink, AttributeEvent event, Object processedValue) refactored to AbstractProtocol.doLinkedAttributeWrite(U agentLink, AttributeEvent event, Object processedValue) as AttributeEvent now contains all required data
  • AbstractIOClientProtocol.createWriteMessage(Attribute<?> attribute, DefaultAgentLink agentLink, AttributeEvent event, Object processedValue) refactored to AbstractIOClientProtocol.createWriteMessage(DefaultAgentLink agentLink, AttributeEvent event, Object processedValue)
  • ProtocolUtil.bytesFromHexString(msg) and ProtocolUtil.bytesFromBinaryString(msg) moved to ValueUtil
  • AssetEvent.getAssetId() refactored to AssetEvent.getId()
  • Frontend code that uses AttributeEvents now must use ref and value instead of attributeState.ref and attributeState.value

If using custom project template then the following changes are also needed:

  • Add the following to gradle.properties:
typescriptGeneratorVersion = 3.2.1263
  • Change settings.gradle to:
plugins {
    id "com.cherryperry.gradle-file-encrypt" version "2.0.3" apply false
    id "org.jetbrains.gradle.plugin.idea-ext" version "1.1.3" apply false
    id 'cz.habarta.typescript-generator' version "$typescriptGeneratorVersion" apply false
}

rootProject.name = "$projectName"

// Include sub-projects dynamically, every directory with a build.gradle (and no .buildignore)
fileTree(dir: rootDir, include: "**/build.gradle", excludes: ["**/node_modules/**", "**/generic_app/**"])
        .filter { it.parent != rootDir }
        .filter { !file("${it.parent}/.buildignore").exists() }
        .each {
            include it.parent.replace(rootDir.canonicalPath, "").replace("\\", ":").replace("/", ":")
        }
2 Likes
13

The latest keycloak image (23.x) has introduced a breaking change which needs to be fixed before upgrading; please run the following whilst the old container is still running, replacing or-postgresql-1 with whatever name you have assigned to your postgresql container:

docker exec or-postgresql-1 psql -U postgres -d openremote -c "INSERT INTO public.client (id, client_id, not_before, realm_id) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', 'account-console', 0, (select id from public.realm where name = 'master')) ON CONFLICT DO NOTHING;
UPDATE public.client set base_url='/realms/master/account/', enabled=true, public_client=true, secret=null, protocol='openid-connect', frontchannel_logout=false, name='\${client_account-console}', client_authenticator_type='client-secret', root_url='\${authBaseUrl}' where id = '2347d8eb-f91a-48b1-98aa-0e631ea0f640';
INSERT INTO public.redirect_uris (client_id, value) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', '/realms/master/account/*') ON CONFLICT DO NOTHING;
INSERT INTO public.client_attributes (client_id, name, value) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', 'pkce.code.challenge.method', 'S256') ON CONFLICT DO NOTHING;
INSERT INTO public.client_attributes (client_id, name, value) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', 'post.logout.redirect.uris', '+') ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'acr' limit 1), true) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'phone' limit 1), false) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'address' limit 1), false) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'microprofile-jwt' limit 1), false) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'offline_access' limit 1), false) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'email' limit 1), true) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'roles' limit 1), true) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'web-origins' limit 1), true) ON CONFLICT DO NOTHING;
INSERT INTO public.client_scope_client (client_id, scope_id, default_scope) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT id from public.client_scope where name = 'profile' limit 1), true) ON CONFLICT DO NOTHING;
INSERT INTO public.protocol_mapper (id, name, protocol, protocol_mapper_name, client_id, client_scope_id) VALUES ('d4e959a8-56a7-4e70-afde-36ac315c228c', 'audience resolve', 'openid-connect', 'oidc-audience-resolve-mapper', '2347d8eb-f91a-48b1-98aa-0e631ea0f640', null) ON CONFLICT DO NOTHING;
INSERT INTO public.scope_mapping (client_id, role_id) VALUES ('2347d8eb-f91a-48b1-98aa-0e631ea0f640', (SELECT kr.id from public.keycloak_role kr inner join public.realm r on r.id = kr.realm_id where kr.name = 'manage-account' and r.name = 'master')) ON CONFLICT DO NOTHING;"

After this you can start using the 23.x image(s) but note any theme customisations should be checked for breaking changes also.

1 Like