Search code examples
dockernginxdocker-composegatewayeclipse-ditto

Eclipse Ditto - 502 Gateway - Clean Deployment - Any hint what needs to be configured?


Following the Ditto Deployment Docker description. E.g. Adding a new password and then executing Docker compose command. Starting with clean Linuxmint 19.3 and new installed Docker and Docker-Compose setup. The localhost:8080 is reachable but fails to load API definition (Fetch error). Additionally I tried Hello World Example - Creating a Thing with adjusted Password set by me and I receive the 502 Gateway. In another Stackoverflow question 502 Bad Gateway nginx/1.13.12 on localhost while creating a new policy in ditto there was mentioned to check docker-compose ps therefore I tried to attach everything in the screenshot. As I said this is clean install Linuxmint 19.3 (Linuxkernel: 5.0.0-37-generic x86_64, + Docker (Docker version 19.03.5) + Docker-Compose (docker-compose version 1.25.0) and then followed the deployment instructions. Anyone has some hints what need to be configured or where I can find more information to that issue? Thank you!

enter image description here

[EDIT]: 27/12/2019 Ubuntu 1910, 502 Gateway with Postman Ditto Profile Setup - clean Ubuntu 1910 + Docker + Docker-compose - services are up and running -> Do not get the Fetch error in the browser anymore but still getting 502 Gateway from Nginx when trying to use the REST-API (using Postman - Ditto Profile) -> See Screenshot (e.g. Creating Things ->still using the Hello World Example).

[EDIT: 02.01.2020] NGINX-LOG:

 {"log":"172.18.0.1 - - [26/Dec/2019:21:55:55 +0000] \"GET / HTTP/1.1\" 200 1555 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:55.043489292Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:55 +0000] \"GET /images/favicon-16x16.png HTTP/1.1\" 200 3082 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:55.387475066Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:55 +0000] \"GET /images/favicon-96x96.png HTTP/1.1\" 200 5612 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:55.395068124Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:57 +0000] \"GET /apidoc/2 HTTP/1.1\" 302 162 \"http://localhost:8080/\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:57.906439165Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:57 +0000] \"GET /apidoc/?url=/apidoc/openapi/ditto-api-2.yml HTTP/1.1\" 200 3465 \"http://localhost:8080/\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:57.955604805Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:58 +0000] \"GET /apidoc/swagger-ui.css HTTP/1.1\" 200 153569 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:58.157003575Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:58 +0000] \"GET /apidoc/swagger-ui-bundle.js HTTP/1.1\" 200 941594 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:58.174659592Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:58 +0000] \"GET /apidoc/swagger-ui-standalone-preset.js HTTP/1.1\" 200 327668 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:58.268308585Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:58 +0000] \"GET /apidoc/images/favicon-96x96.png HTTP/1.1\" 200 5612 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:58.982532434Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:58 +0000] \"GET /apidoc/images/favicon-16x16.png HTTP/1.1\" 200 3082 \"-\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:58.983939986Z"}
{"log":"172.18.0.1 - - [26/Dec/2019:21:55:59 +0000] \"GET /apidoc/openapi/ditto-api-2.yml HTTP/1.1\" 200 181103 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2019-12-26T21:55:59.56570392Z"}
{"log":"172.18.0.1 - - [02/Jan/2020:08:42:09 +0000] \"GET /apidoc/?url=/apidoc/openapi/ditto-api-2.yml HTTP/1.1\" 304 0 \"http://localhost:8080/\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2020-01-02T08:42:09.304290027Z"}
{"log":"172.18.0.1 - - [02/Jan/2020:08:42:09 +0000] \"GET /apidoc/swagger-ui.css HTTP/1.1\" 304 0 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2020-01-02T08:42:09.552048529Z"}
{"log":"172.18.0.1 - - [02/Jan/2020:08:42:09 +0000] \"GET /apidoc/swagger-ui-standalone-preset.js HTTP/1.1\" 304 0 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2020-01-02T08:42:09.571606549Z"}
{"log":"172.18.0.1 - - [02/Jan/2020:08:42:09 +0000] \"GET /apidoc/swagger-ui-bundle.js HTTP/1.1\" 304 0 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2020-01-02T08:42:09.571997943Z"}
{"log":"172.18.0.1 - - [02/Jan/2020:08:42:10 +0000] \"GET /apidoc/openapi/ditto-api-2.yml HTTP/1.1\" 304 0 \"http://localhost:8080/apidoc/?url=/apidoc/openapi/ditto-api-2.yml\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0\"\n","stream":"stdout","time":"2020-01-02T08:42:10.827640415Z"}
{"log":"2020/01/02 09:06:04 [error] 6#6: *17 connect() failed (113: Host is unreachable) while connecting to upstream, client: 172.18.0.1, server: localhost, request: \"PUT /api/1/things/org.eclipse.ditto:fancy-car HTTP/1.1\", upstream: \"http://172.18.0.8:8080/api/1/things/org.eclipse.ditto:fancy-car\", host: \"localhost:8080\"\n","stream":"stderr","time":"2020-01-02T09:06:04.708448484Z"}
{"log":"172.18.0.1 - ditto [02/Jan/2020:09:06:04 +0000] \"PUT /api/1/things/org.eclipse.ditto:fancy-car HTTP/1.1\" 502 174 \"-\" \"curl/7.65.3\"\n","stream":"stdout","time":"2020-01-02T09:06:04.710901566Z"}

LOG Gateway:

{"log":"10:35:13,631 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set\n","stream":"stdout","time":"2020-01-02T09:35:15.303215853Z"}
{"log":"10:35:13,664 |-INFO in ch.qos.logback.core.joran.action.StatusListenerAction - Added status listener of type [ch.qos.logback.core.status.OnConsoleStatusListener]\n","stream":"stdout","time":"2020-01-02T09:35:15.303240849Z"}
{"log":"10:35:15,300 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]\n","stream":"stdout","time":"2020-01-02T09:35:15.303245298Z"}
{"log":"10:35:15,585 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]\n","stream":"stdout","time":"2020-01-02T09:35:15.585392416Z"}
{"log":"10:35:15,962 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property\n","stream":"stdout","time":"2020-01-02T09:35:15.962866673Z"}
{"log":"10:35:17,657 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]\n","stream":"stdout","time":"2020-01-02T09:35:17.663092596Z"}
{"log":"10:35:17,657 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDERR]\n","stream":"stdout","time":"2020-01-02T09:35:17.663114435Z"}
{"log":"10:35:17,658 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property\n","stream":"stdout","time":"2020-01-02T09:35:17.756369134Z"}
{"log":"10:35:18,018 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.mongodb.driver] to WARN\n","stream":"stdout","time":"2020-01-02T09:35:18.018757693Z"}
{"log":"10:35:18,037 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] to WARN\n","stream":"stdout","time":"2020-01-02T09:35:18.038838491Z"}
{"log":"10:35:18,038 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] to WARN\n","stream":"stdout","time":"2020-01-02T09:35:18.038855621Z"}
{"log":"10:35:18,089 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.eclipse.ditto] to INFO\n","stream":"stdout","time":"2020-01-02T09:35:18.091138646Z"}
{"log":"10:35:18,090 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO\n","stream":"stdout","time":"2020-01-02T09:35:18.091165529Z"}
{"log":"10:35:18,090 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]\n","stream":"stdout","time":"2020-01-02T09:35:18.091168939Z"}
{"log":"10:35:18,163 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDERR] to Logger[ROOT]\n","stream":"stdout","time":"2020-01-02T09:35:18.166843406Z"}
{"log":"10:35:18,168 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.\n","stream":"stdout","time":"2020-01-02T09:35:18.169865148Z"}
{"log":"10:35:18,232 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@7c4f4c56 - Registering current configuration as safe fallback point\n","stream":"stdout","time":"2020-01-02T09:35:18.235105545Z"}
{"log":"2020-01-02 10:35:18,628 INFO  [] o.e.d.s.u.c.r.ServiceSpecificEnvironmentConfigSupplier  - Running in \u003cDOCKER\u003e environment.\n","stream":"stdout","time":"2020-01-02T09:35:18.666136956Z"}
{"log":"2020-01-02 10:35:48,250 INFO  [] o.e.d.s.g.s.GatewayService  - Running with following runtime parameters: \u003c[-Xoptionsfile=/opt/java/openjdk/jre/lib/amd64/compressedrefs/options.default, -Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple, -Xjcl:jclse7b_29, -Dcom.ibm.oti.vm.bootstrap.library.path=/opt/java/openjdk/jre/lib/amd64/compressedrefs:/opt/java/openjdk/jre/lib/amd64, -Dsun.boot.library.path=/opt/java/openjdk/jre/lib/amd64/compressedrefs:/opt/java/openjdk/jre/lib/amd64, -Djava.library.path=/opt/java/openjdk/jre/lib/amd64/compressedrefs:/opt/java/openjdk/jre/lib/amd64:/usr/lib64:/usr/lib, -Djava.home=/opt/java/openjdk/jre, -Djava.ext.dirs=/opt/java/openjdk/jre/lib/ext, -Duser.dir=/opt/ditto, -XX:+IgnoreUnrecognizedVMOptions, -XX:+UseContainerSupport, -XX:+IdleTuningCompactOnIdle, -XX:+IdleTuningGcOnIdle, -XX:+ExitOnOutOfMemoryError, -Xtune:virtualized, -Xss512k, -XX:MaxRAMPercentage=80, -Dakka.coordinated-shutdown.exit-jvm=on, -Dakka.cluster.shutdown-after-unsuccessful-join-seed-nodes=120s, -Djava.class.path=., -Dditto.gateway.authentication.devops.password=foobar, -Djava.class.path=starter.jar, -Dsun.java.command=starter.jar, -Dsun.java.launcher=SUN_STANDARD, -Dsun.java.launcher.pid=6]\u003e.\n","stream":"stdout","time":"2020-01-02T09:35:48.267097485Z"}
{"log":"2020-01-02 10:35:48,263 INFO  [] o.e.d.s.g.s.GatewayService  - Available processors: \u003c1\u003e.\n","stream":"stdout","time":"2020-01-02T09:35:48.267125489Z"}
{"log":"2020-01-02 10:36:03,043 INFO  [] k.m.SystemMetrics  - Starting the Kamon(SystemMetrics) module\n","stream":"stdout","time":"2020-01-02T09:36:03.046833797Z"}
{"log":"2020-01-02 10:36:06,217 INFO  [] o.e.d.s.g.s.GatewayService  - Successfully added Prometheus reporter to Kamon.\n","stream":"stdout","time":"2020-01-02T09:36:06.438978998Z"}
{"log":"2020-01-02 10:36:32,106 INFO  [] a.e.s.Slf4jLogger  - Slf4jLogger started\n","stream":"stdout","time":"2020-01-02T09:36:32.115707385Z"}
{"log":"2020-01-02 10:36:59,770 ERROR [] o.e.d.s.g.s.GatewayService  - An exception occurred in main method of \u003corg.eclipse.ditto.services.gateway.starter.GatewayService\u003e!\n","stream":"stdout","time":"2020-01-02T09:37:00.243987944Z"}
{"log":"java.util.concurrent.TimeoutException: Futures timed out after [3000 milliseconds]\n","stream":"stdout","time":"2020-01-02T09:37:00.244013123Z"}
{"log":"\u0009at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)\n","stream":"stdout","time":"2020-01-02T09:37:00.244016915Z"}
{"log":"\u0009at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)\n","stream":"stdout","time":"2020-01-02T09:37:00.244032597Z"}
{"log":"\u0009at scala.concurrent.Await$.$anonfun$result$1(package.scala:220)\n","stream":"stdout","time":"2020-01-02T09:37:00.244035603Z"}
{"log":"\u0009at scala.concurrent.Await$$$Lambda$273.00000000E15AF3D0.apply(Unknown Source)\n","stream":"stdout","time":"2020-01-02T09:37:00.244038575Z"}
{"log":"\u0009at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)\n","stream":"stdout","time":"2020-01-02T09:37:00.244041572Z"}
{"log":"\u0009at scala.concurrent.Await$.result(package.scala:146)\n","stream":"stdout","time":"2020-01-02T09:37:00.244044582Z"}
{"log":"\u0009at akka.remote.artery.tcp.ArteryTcpTransport.bindInboundStreams(ArteryTcpTransport.scala:238)\n","stream":"stdout","time":"2020-01-02T09:37:00.244047506Z"}
{"log":"\u0009at akka.remote.artery.ArteryTransport.start(ArteryTransport.scala:449)\n","stream":"stdout","time":"2020-01-02T09:37:00.24405049Z"}
{"log":"\u0009at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:234)\n","stream":"stdout","time":"2020-01-02T09:37:00.244053454Z"}
{"log":"\u0009at akka.cluster.ClusterActorRefProvider.init(ClusterActorRefProvider.scala:40)\n","stream":"stdout","time":"2020-01-02T09:37:00.244056387Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:912)\n","stream":"stdout","time":"2020-01-02T09:37:00.244059323Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:908)\n","stream":"stdout","time":"2020-01-02T09:37:00.244062242Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl._start(ActorSystem.scala:908)\n","stream":"stdout","time":"2020-01-02T09:37:00.244065174Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl.start(ActorSystem.scala:930)\n","stream":"stdout","time":"2020-01-02T09:37:00.244068096Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:258)\n","stream":"stdout","time":"2020-01-02T09:37:00.24407099Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:302)\n","stream":"stdout","time":"2020-01-02T09:37:00.244073925Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:276)\n","stream":"stdout","time":"2020-01-02T09:37:00.244076814Z"}
{"log":"\u0009at akka.actor.ActorSystem$.create(ActorSystem.scala:199)\n","stream":"stdout","time":"2020-01-02T09:37:00.244079716Z"}
{"log":"\u0009at akka.actor.ActorSystem.create(ActorSystem.scala)\n","stream":"stdout","time":"2020-01-02T09:37:00.24408264Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.createActorSystem(DittoService.java:339)\n","stream":"stdout","time":"2020-01-02T09:37:00.244085545Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.doStart(DittoService.java:223)\n","stream":"stdout","time":"2020-01-02T09:37:00.244088552Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService$$Lambda$8.00000000E0D6E4D0.call(Unknown Source)\n","stream":"stdout","time":"2020-01-02T09:37:00.244091578Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.tryToRunMainMethodBody(MainMethodExceptionHandler.java:70)\n","stream":"stdout","time":"2020-01-02T09:37:00.244094596Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.call(MainMethodExceptionHandler.java:64)\n","stream":"stdout","time":"2020-01-02T09:37:00.244111097Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.start(DittoService.java:207)\n","stream":"stdout","time":"2020-01-02T09:37:00.244114634Z"}
{"log":"\u0009at org.eclipse.ditto.services.gateway.starter.GatewayService.main(GatewayService.java:52)\n","stream":"stdout","time":"2020-01-02T09:37:00.24411729Z"}
{"log":"2020-01-02 10:36:59,770 ERROR [] o.e.d.s.g.s.GatewayService  - An exception occurred in main method of \u003corg.eclipse.ditto.services.gateway.starter.GatewayService\u003e!\n","stream":"stderr","time":"2020-01-02T09:37:00.630365422Z"}
{"log":"java.util.concurrent.TimeoutException: Futures timed out after [3000 milliseconds]\n","stream":"stderr","time":"2020-01-02T09:37:00.630386475Z"}
{"log":"\u0009at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)\n","stream":"stderr","time":"2020-01-02T09:37:00.630389802Z"}
{"log":"\u0009at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)\n","stream":"stderr","time":"2020-01-02T09:37:00.630392537Z"}
{"log":"\u0009at scala.concurrent.Await$.$anonfun$result$1(package.scala:220)\n","stream":"stderr","time":"2020-01-02T09:37:00.630395183Z"}
{"log":"\u0009at scala.concurrent.Await$$$Lambda$273.00000000E15AF3D0.apply(Unknown Source)\n","stream":"stderr","time":"2020-01-02T09:37:00.630397814Z"}
{"log":"\u0009at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)\n","stream":"stderr","time":"2020-01-02T09:37:00.63040037Z"}
{"log":"\u0009at scala.concurrent.Await$.result(package.scala:146)\n","stream":"stderr","time":"2020-01-02T09:37:00.630403008Z"}
{"log":"\u0009at akka.remote.artery.tcp.ArteryTcpTransport.bindInboundStreams(ArteryTcpTransport.scala:238)\n","stream":"stderr","time":"2020-01-02T09:37:00.630405552Z"}
{"log":"\u0009at akka.remote.artery.ArteryTransport.start(ArteryTransport.scala:449)\n","stream":"stderr","time":"2020-01-02T09:37:00.630408131Z"}
{"log":"\u0009at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:234)\n","stream":"stderr","time":"2020-01-02T09:37:00.630410669Z"}
{"log":"\u0009at akka.cluster.ClusterActorRefProvider.init(ClusterActorRefProvider.scala:40)\n","stream":"stderr","time":"2020-01-02T09:37:00.630413248Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:912)\n","stream":"stderr","time":"2020-01-02T09:37:00.630415792Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:908)\n","stream":"stderr","time":"2020-01-02T09:37:00.630418351Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl._start(ActorSystem.scala:908)\n","stream":"stderr","time":"2020-01-02T09:37:00.630420915Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl.start(ActorSystem.scala:930)\n","stream":"stderr","time":"2020-01-02T09:37:00.630423489Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:258)\n","stream":"stderr","time":"2020-01-02T09:37:00.630426023Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:302)\n","stream":"stderr","time":"2020-01-02T09:37:00.630428584Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:276)\n","stream":"stderr","time":"2020-01-02T09:37:00.63043109Z"}
{"log":"\u0009at akka.actor.ActorSystem$.create(ActorSystem.scala:199)\n","stream":"stderr","time":"2020-01-02T09:37:00.630433609Z"}
{"log":"\u0009at akka.actor.ActorSystem.create(ActorSystem.scala)\n","stream":"stderr","time":"2020-01-02T09:37:00.630436124Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.createActorSystem(DittoService.java:339)\n","stream":"stderr","time":"2020-01-02T09:37:00.630438662Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.doStart(DittoService.java:223)\n","stream":"stderr","time":"2020-01-02T09:37:00.630441298Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService$$Lambda$8.00000000E0D6E4D0.call(Unknown Source)\n","stream":"stderr","time":"2020-01-02T09:37:00.630443891Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.tryToRunMainMethodBody(MainMethodExceptionHandler.java:70)\n","stream":"stderr","time":"2020-01-02T09:37:00.630455503Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.call(MainMethodExceptionHandler.java:64)\n","stream":"stderr","time":"2020-01-02T09:37:00.630458303Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.start(DittoService.java:207)\n","stream":"stderr","time":"2020-01-02T09:37:00.630461624Z"}
{"log":"\u0009at org.eclipse.ditto.services.gateway.starter.GatewayService.main(GatewayService.java:52)\n","stream":"stderr","time":"2020-01-02T09:37:00.630464264Z"}
{"log":"Exception in thread \"main\" java.lang.RuntimeException: Exception on main thread: Futures timed out after [3000 milliseconds]\n","stream":"stderr","time":"2020-01-02T09:37:00.630466883Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.tryToRunMainMethodBody(MainMethodExceptionHandler.java:77)\n","stream":"stderr","time":"2020-01-02T09:37:00.630469646Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.call(MainMethodExceptionHandler.java:64)\n","stream":"stderr","time":"2020-01-02T09:37:00.630472255Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.start(DittoService.java:207)\n","stream":"stderr","time":"2020-01-02T09:37:00.63047486Z"}
{"log":"\u0009at org.eclipse.ditto.services.gateway.starter.GatewayService.main(GatewayService.java:52)\n","stream":"stderr","time":"2020-01-02T09:37:00.684763121Z"}
{"log":"Caused by: java.util.concurrent.TimeoutException: Futures timed out after [3000 milliseconds]\n","stream":"stderr","time":"2020-01-02T09:37:00.684781666Z"}
{"log":"\u0009at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)\n","stream":"stderr","time":"2020-01-02T09:37:00.684784878Z"}
{"log":"\u0009at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)\n","stream":"stderr","time":"2020-01-02T09:37:00.684787498Z"}
{"log":"\u0009at scala.concurrent.Await$.$anonfun$result$1(package.scala:220)\n","stream":"stderr","time":"2020-01-02T09:37:00.684790021Z"}
{"log":"\u0009at scala.concurrent.Await$$$Lambda$273.00000000E15AF3D0.apply(Unknown Source)\n","stream":"stderr","time":"2020-01-02T09:37:00.684792625Z"}
{"log":"\u0009at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)\n","stream":"stderr","time":"2020-01-02T09:37:00.684795141Z"}
{"log":"\u0009at scala.concurrent.Await$.result(package.scala:146)\n","stream":"stderr","time":"2020-01-02T09:37:00.684797658Z"}
{"log":"\u0009at akka.remote.artery.tcp.ArteryTcpTransport.bindInboundStreams(ArteryTcpTransport.scala:238)\n","stream":"stderr","time":"2020-01-02T09:37:00.684800149Z"}
{"log":"\u0009at akka.remote.artery.ArteryTransport.start(ArteryTransport.scala:449)\n","stream":"stderr","time":"2020-01-02T09:37:00.684802666Z"}
{"log":"\u0009at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:234)\n","stream":"stderr","time":"2020-01-02T09:37:00.684805127Z"}
{"log":"\u0009at akka.cluster.ClusterActorRefProvider.init(ClusterActorRefProvider.scala:40)\n","stream":"stderr","time":"2020-01-02T09:37:00.684807611Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:912)\n","stream":"stderr","time":"2020-01-02T09:37:00.684810102Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:908)\n","stream":"stderr","time":"2020-01-02T09:37:00.684812574Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl._start(ActorSystem.scala:908)\n","stream":"stderr","time":"2020-01-02T09:37:00.684815067Z"}
{"log":"\u0009at akka.actor.ActorSystemImpl.start(ActorSystem.scala:930)\n","stream":"stderr","time":"2020-01-02T09:37:00.684817544Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:258)\n","stream":"stderr","time":"2020-01-02T09:37:00.684820019Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:302)\n","stream":"stderr","time":"2020-01-02T09:37:00.684822513Z"}
{"log":"\u0009at akka.actor.ActorSystem$.apply(ActorSystem.scala:276)\n","stream":"stderr","time":"2020-01-02T09:37:00.907482608Z"}
{"log":"\u0009at akka.actor.ActorSystem$.create(ActorSystem.scala:199)\n","stream":"stderr","time":"2020-01-02T09:37:00.907504877Z"}
{"log":"\u0009at akka.actor.ActorSystem.create(ActorSystem.scala)\n","stream":"stderr","time":"2020-01-02T09:37:00.93876599Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.createActorSystem(DittoService.java:339)\n","stream":"stderr","time":"2020-01-02T09:37:00.93878602Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService.doStart(DittoService.java:223)\n","stream":"stderr","time":"2020-01-02T09:37:00.938789392Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.DittoService$$Lambda$8.00000000E0D6E4D0.call(Unknown Source)\n","stream":"stderr","time":"2020-01-02T09:37:00.938792095Z"}
{"log":"\u0009at org.eclipse.ditto.services.base.MainMethodExceptionHandler.tryToRunMainMethodBody(MainMethodExceptionHandler.java:70)\n","stream":"stderr","time":"2020-01-02T09:37:00.938794664Z"}
{"log":"\u0009... 3 more\n","stream":"stderr","time":"2020-01-02T09:37:00.9387973Z"}
{"log":"2020-01-02 10:37:01,302 INFO  [] a.r.RemoteActorRefProvider$RemotingTerminator akka://ditto-cluster/system/remoting-terminator - Shutting down remote daemon.\n","stream":"stdout","time":"2020-01-02T09:37:01.318416114Z"}
{"log":"2020-01-02 10:37:01,676 INFO  [] a.r.RemoteActorRefProvider$RemotingTerminator akka://ditto-cluster/system/remoting-terminator - Remote daemon shut down; proceeding with flushing remote transports.\n","stream":"stdout","time":"2020-01-02T09:37:01.689865086Z"}
{"log":"2020-01-02 10:37:03,007 INFO  [] a.r.RemoteActorRefProvider$RemotingTerminator akka://ditto-cluster/system/remoting-terminator - Remoting shut down.\n","stream":"stdout","time":"2020-01-02T09:37:03.01095963Z"}

Solution

  • That sounds like a challenging problem to solve. Based on the logs of the Ditto gateway service binding the Akka Artery inbound TCP streams times out:

    Caused by: java.util.concurrent.TimeoutException: Futures timed out after [3000 milliseconds]
      at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)
      at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)
      at scala.concurrent.Await$.$anonfun$result$1(package.scala:220)
      at scala.concurrent.Await$$$Lambda$273.00000000E15AF3D0.apply(Unknown Source)
      at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)
      at scala.concurrent.Await$.result(package.scala:146)
      at akka.remote.artery.tcp.ArteryTcpTransport.bindInboundStreams(ArteryTcpTransport.scala:238)
      at akka.remote.artery.ArteryTransport.start(ArteryTransport.scala:449)
      at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:234)
    
    

    I have no idea what could be special in your setup that makes this happen. My only wild guess would be: How many CPU cores do you have available for your Docker daemon?

    Try to increase the CPU resources you run the Ditto services with. From my experience you need at least 4 CPU cores in total so that a Ditto cluster can form itself reliably.