Search code examples
kuberneteskubernetes-helmartifactoryazure-aksjfrog-xray

Jfrog Xray Helm deployment failing on Azure Kubernetes Serivice


I have deployed Artifactory on Azure Kubernetes Service using the official Jfrog chart: https://github.com/jfrog/charts/tree/master/stable/artifactory

Then I tried to deploy Xray using the chart: https://github.com/jfrog/charts/tree/master/stable/xray

helm upgrade --install xray --set xray.joinKey=50f330cfed7eea5947f5423323146c03 --set xray.jfrogUrl=http://X.X.X.X --namespace xray jfrog/xray

Xray deployment is able to join to the cluster however, Only xray-0 Pod is failing with the status=CrashLoopBackOff

Error Logs

2022-06-29T17:03:50.372Z [jfrou] [INFO ] [20d064dc265f5e6d] [bootstrap.go:88               ] [main                ] - Router (jfrou) service initialization started. Version: 7.38.0-1 Revision: 89f07fc2b65bad0638539945b1ea3ba9017b068b PID: 230 Home: /opt/jfrog/router
2022-06-29T17:03:50.374Z [jfrou] [INFO ] [20d064dc265f5e6d] [bootstrap.go:91               ] [main                ] - JFrog Router IP: 10.1.1.49
2022-06-29T17:03:50.375Z [jfrou] [INFO ] [20d064dc265f5e6d] [bootstrap.go:204              ] [main                ] - System configuration encryption report:
shared.newrelic.licenseKey: does not exist in the config file
shared.security.joinKeyFile: file '/opt/jfrog/router/var/etc/security/join.key' - already encrypted
2022-06-29T17:03:50.375Z [jfrou] [INFO ] [20d064dc265f5e6d] [bootstrap.go:96               ] [main                ] - JFrog Router Service ID: jfrou@01g6q7hrg1hac569r9e27yjrxe
2022-06-29T17:03:50.375Z [jfrou] [INFO ] [20d064dc265f5e6d] [bootstrap.go:97               ] [main                ] - JFrog Router Node ID: xray-0
2022-06-29T17:03:50.390Z [jfrou] [INFO ] [20d064dc265f5e6d] [trusted.go:36                 ] [main                ] - System cert pool contents were loaded as trusted CAs for TLS communication
2022-06-29T17:03:50.390Z [jfrou] [INFO ] [20d064dc265f5e6d] [trusted.go:56                 ] [main                ] - Following certificates were successfully loaded as trusted CAs for TLS communication:
[/opt/jfrog/router/var/data/router/keys/trusted/access-root-ca.crt]
2022-06-29T17:03:50.390Z [jfrou] [INFO ] [20d064dc265f5e6d] [trusted.go:36                 ] [main                ] - System cert pool contents were loaded as trusted CAs for TLS communication
2022-06-29T17:03:50.390Z [jfrou] [INFO ] [20d064dc265f5e6d] [trusted.go:56                 ] [main                ] - Following certificates were successfully loaded as trusted CAs for TLS communication:
[/opt/jfrog/router/var/data/router/keys/trusted/access-root-ca.crt]
2022-06-29T17:03:50.472Z [jfrou] [INFO ] [20d064dc265f5e6d] [join_executor.go:117          ] [main                ] - Cluster join: Trying to rejoin the cluster
2022-06-29T17:03:50.525Z [jfrou] [INFO ] [20d064dc265f5e6d] [join_executor.go:206          ] [main                ] - Cluster join: Successfully joined the cluster
2022-06-29T17:03:50.527Z [jfrou] [INFO ] [20d064dc265f5e6d] [trusted.go:36                 ] [main                ] - System cert pool contents were loaded as trusted CAs for TLS communication
2022-06-29T17:03:50.527Z [jfrou] [INFO ] [20d064dc265f5e6d] [trusted.go:56                 ] [main                ] - Following certificates were successfully loaded as trusted CAs for TLS communication:
[/opt/jfrog/router/var/data/router/keys/trusted/access-root-ca.crt]
2022-06-29T17:03:50.538Z [jfrou] [INFO ] [20d064dc265f5e6d] [mtls_client.go:221            ] [main                ] - Detected mTLS configuration change, resetting users and certificates cache expiry to 1m40s
2022-06-29T17:03:50.540Z [jfrou] [INFO ] [20d064dc265f5e6d] [registry_handler.go:77        ] [main                ] - Persisted local services: []
2022-06-29T17:03:50.542Z [jfrou] [INFO ] [                ] [logger.go:58                  ] [main                ] - Starting provider aggregator.ProviderAggregator {}
2022-06-29T17:03:50.543Z [jfrou] [INFO ] [                ] [logger.go:58                  ] [main                ] - Starting provider *file.Provider {"directory":"/opt/jfrog/router/var/data/router/traefik/routes","watch":true}
2022-06-29T17:03:50.544Z [jfrou] [INFO ] [                ] [logger.go:58                  ] [main                ] - Starting provider *traefik.Provider {}
2022-06-29T17:03:50.678Z [jfrou] [INFO ] [20d064dc265f5e6d] [local_topology.go:116         ] [main                ] - Required service types: [jfxr jfxana jfxidx jfxpst jfob]
2022-06-29T17:03:50.678Z [jfrou] [INFO ] [20d064dc265f5e6d] [main.go:49                    ] [main                ] - Router (jfrou) service initialization completed in 0.307 seconds. Listening on port: 8082
2022-06-29T17:03:50.678Z [jfrou] [WARN ] [0853b27442453374] [local_topology_helper.go:65   ] [main                ] - Missing required services: [jfxr jfxana jfxidx jfxpst jfob]
2022-06-29T17:06:45.699Z [jfrou] [WARN ] [3bb36f6cd428c3a6] [local_topology.go:274         ] [main                ] - Readiness test failed with the following error: "required node services are missing or unhealthy"
2022-06-29T17:06:47.720Z [jfrou] [WARN ] [7dbaaf720b7399c6] [external_topology.go:126      ] [main                ] - restart needed, the node has no registered services
2022-06-29T17:06:48.771Z [jfrou] [WARN ] [0a7f307c1e6bffbb] [local_topology_helper.go:65   ] [main                ] - Missing required services: [jfxr jfxana jfxidx jfxpst jfob]
Terminate signal received, attempting a graceful shutdown
2022-06-29T17:06:49.323Z [jfrou] [INFO ] [20d064dc265f5e6d] [application.go:72             ] [main                ] - Received OS signal: terminated
2022-06-29T17:06:49.323Z [jfrou] [INFO ] [20d064dc265f5e6d] [main.go:54                    ] [main                ] - Shutting down
Checking if process is alive (attempt #1 / 30)...
2022-06-29T17:06:49.328Z [jfrou] [INFO ] [                ] [logger.go:90                  ] [main                ] - I have to go...
2022-06-29T17:06:49.328Z [jfrou] [INFO ] [                ] [logger.go:90                  ] [main                ] - Stopping server gracefully
2022-06-29T17:06:49.328Z [jfrou] [ERROR] [                ] [server_entrypoint_tcp.go:188  ] [main                ] - accept tcp 127.0.0.1:8046: use of closed network connection
2022-06-29T17:06:49.328Z [jfrou] [ERROR] [                ] [server_entrypoint_tcp.go:533  ] [main                ] - Error while starting server: accept tcp 127.0.0.1:8046: use of closed network connection
2022-06-29T17:06:49.328Z [jfrou] [ERROR] [                ] [server_entrypoint_tcp.go:188  ] [main                ] - accept tcp [::]:8082: use of closed network connection
2022-06-29T17:06:49.328Z [jfrou] [ERROR] [                ] [server_entrypoint_tcp.go:533  ] [main                ] - Error while starting server: accept tcp [::]:8082: use of closed network connection
2022-06-29T17:06:49.328Z [jfrou] [ERROR] [                ] [server_entrypoint_tcp.go:188  ] [main                ] - accept tcp 127.0.0.1:8049: use of closed network connection
2022-06-29T17:06:49.328Z [jfrou] [ERROR] [                ] [server_entrypoint_tcp.go:533  ] [main                ] - Error while starting server: accept tcp 127.0.0.1:8049: use of closed network connection
Process 230 is still running
2022-06-29T17:06:49.828Z [jfrou] [INFO ] [                ] [logger.go:90                  ] [main                ] - Server stopped
2022-06-29T17:06:49.828Z [jfrou] [INFO ] [                ] [application.go:80             ] [main                ] - JFrog Router stopped

Solution

  • Are you using an IP address in the jfrogurl param? You could try to use the service name of artifactory plus adding the port 8082 to the url. Seems like xray is not registering correctly although the logs may suggest it. What is the Artifactory monitoring page telling you? Xray online or offline?