Search code examples
etltalend

Talend Open Studio - tssh losing connection to main job

Talend Open Studio for Data Integration, V7.2.1.

I am working on an ETL process which uses tssh components, to execute a shell script on a remote server. Once the execution on the remote server completes, it continues the process on the host server.

The talend job seems to work fine if the process on the remote server takes a few minutes, but it hangs up if the process on the remote server takes longer ( more than 1 hour in this case ). The process on host server doesn't continue even after the process on the remote server has completed. It doesn't throw any error but the java process seems to be running indefinitely without any real progress.

I tried the following without any success -

  1. I initially connected the tssh component to a tsystem component using onComponentOk trigger, without any timeout settings.
  2. I added an exit command in the tssh component, after the call to execute the shell script, to try and close the connection.
  3. I added timeout settings to the tssh component and connected tssh to tsytem using Runif trigger while checking the exit_code. ((Integer)globalMap.get("tSSH_1_EXIT_CODE")) == 0

The process seems to be frozen if the tssh takes too long. Any help would be greatly appreciated.

I found a similar thread, but couldn't find a solution - tSSH-Talend blockes job from continuing Run

Thank you

Solution

  • It is possible that the problem is related to the keepalive on the ssh server side.

    Does the connection go through firewalls or vpn?

    You can try to connect with a simple ssh client, and if after an hour you are disconnected having typed nothing at the client, it is because a network equipment closes the connection.

    more info here :) https://unix.stackexchange.com/questions/200239/how-can-i-keep-my-ssh-sessions-from-freezing