Gateway -> EDGE: Gateway connector disconnected

Hi everyone,
First I want to congratulate the OR team for the good work they have been doing with the platform. As we move forward we will try to provide the community with any improvements we can make.

Our focus is Industry 4.0 and we have been testing OR for three months. In the last few days I have been testing OR Edge and I have a disconnection problem around 5 minutes after the connection is established. Could it be a keycloak authorization issue? I attach the logs of the Master and the Edge.

Logs OR Master
manager_1 | 2022-07-29 18:56:12.053 INFO [Messaging-ClientEventQue…e-16] manager.gateway.GatewayConnector.GATEWAY : Gateway connector disconnected: Gateway ID=6BGJqOHCj3NG1F4gvd5Hb5
manager_1 | 2022-07-29 18:57:44.092 INFO [nioEventLoopGroup-3-4 ] org.keycloak.adapters.KeycloakDeployment : Loaded URLs from http://keycloak:8080/auth/realms/labbees/.well-known/openid-configuration

Logs OR EDGE
manager_1 | 2022-07-29 18:56:11.867 INFO [nioEventLoopGroup-7-1 ] otocol.io.AbstractNettyIOClient.PROTOCOL : Connection closed un-expectedly: wss://panel.bees.com.ar:443/websocket/events?Realm=labbees
manager_1 | 2022-07-29 18:56:11.870 WARNING [nioEventLoopGroup-7-1 ] io.netty.util.concurrent.DefaultPromise : An exception was thrown by org.openremote.agent.protocol.io.AbstractNettyIOClient$$Lambda$669/0x000000080169ac98.operationComplete()
manager_1 | java.lang.IllegalStateException: delay must be >= the jitter duration
manager_1 | at dev.failsafe.internal.util.Assert.state(Assert.java:40)
manager_1 | at dev.failsafe.RetryPolicyBuilder.withDelay(RetryPolicyBuilder.java:328)
manager_1 | at org.openremote.agent.protocol.io.AbstractNettyIOClient.scheduleDoConnect(AbstractNettyIOClient.java:224)
manager_1 | at org.openremote.agent.protocol.io.AbstractNettyIOClient.lambda$doConnect$3(AbstractNettyIOClient.java:291)
manager_1 | at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
manager_1 | at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
manager_1 | at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
manager_1 | at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
manager_1 | at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:605)
manager_1 | at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
manager_1 | at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
manager_1 | at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1182)
manager_1 | at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:773)
manager_1 | at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:749)
manager_1 | at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:620)
manager_1 | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:105)
manager_1 | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:174)
manager_1 | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
manager_1 | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
manager_1 | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
manager_1 | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
manager_1 | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
manager_1 | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
manager_1 | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
manager_1 | at java.base/java.lang.Thread.run(Thread.java:833)
manager_1 | 2022-07-29 18:56:11.871 INFO [Scheduled task-21 ] ger.gateway.GatewayClientService.GATEWAY : Connection status change for gateway IO client ‘CONNECTING’: GatewayConnection{localRealm=‘master’, host=‘panel.bees.com.ar’, port=443, realm=‘labbees’, clientId=‘gateway-6bgjqohcj3ng1f4gvd5hb5’, clientSecret=‘4443e0b9-5225-4379-bda3-024e06271afa’, secured=true, disabled=false}

1 Like

Good afternoon, I kept doing tests and I can’t find the problem. Any idea what it can be? Thank you very much.

Good morning, I deleted the containers and made a new pull in the EDGE and the connectivity was solved. The disconnection with the Openremote Main is still there but it performs an automatic reconnection.

Greetings.

1 Like

Hi Mariano,

Thanks for the update on this issue. We didn’t have time to look into it yet, but I will put it on our list to review the stability of this feature.

If you have any more data to share it could be helpful.

Don

Hi Don.

I think it’s a problem with Netty on the EDGE side. after 6 minutes it sends the following error:

manager_1 | 2022-07-29 18:56:11.870 WARNING [nioEventLoopGroup-7-1 ] io.netty.util.concurrent.DefaultPromise : An exception was thrown by org.openremote.agent.protocol.io.AbstractNettyIOClient$$Lambda$669/0x000000080169ac98.operationComplete()
manager_1 | java.lang.IllegalStateException: delay must be >= the jitter duration

It disconnects and reconnects. I’ll keep looking and if I find anything I’ll post it.

Greetings.

Hi,

I would not expect the error you have posted to appear since a code change in June; can you confirm what version of the containers you are running, assuming you have the latest tag of the manager image:

docker image inspect openremote/manager:latest

Thanks.

Hi Rich,

Actually it was solved with the update to the latest version. At this moment we are testing with 4 EDGE and it works correctly.

Thanks for the info!!!

Greetings.