Search code examples
continuous-integrationgitlabexpectspawn

A local run of expect script works fine, but hangs when ran through gitlab CI

I'm writing an expect file to connect to handle the interaction with the remote device by using a script that looks something like the following, and I am facing a few problems with it. Mainly, when I run the script stand-alone or interactively (on a runner), it works as I wish, but when I plug in the code for a testing using the Gitlab CI on one of the runners, it has the following problems:

  1. the command spawn /usr/bin/scp $rsaFile remote:/var/root/id_rsa.pub doesn't seem to show up in the CI log (though the first spawn /usr/bin/ssh remote command shows up) - I suspect it's not being called at all.
  2. The same spawn command persistently gives me the following message until the script stops executing at the next interact command: "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\n"
  3. interact complains that spawn id exp0 not open (which I assume may be because scp command not working properly)

Here's the code:

spawn /usr/bin/ssh remote

expect {
    -re ".*assword:.*" {
        exp_send "password\r"; 
        exp_continue
    }
    -re ".*sh.*" {
        exp_send "mount -uw /\r";
        exp_send "nvram -somecommand"
        exp_send "exit\r"
    }
}

interact

# Copy over the key
set rsaFile "/var/root/Resources/id_rsa.pub"
spawn /usr/bin/scp $rsaFile remote:/var/root/id_rsa.pub;

expect {
    # 1&2 were added to the original script for debugging purposes
    -re "^id_rsa.pub.*" { #1: output I'd expect in case of successful scp
        exp_send "exit\r"
    }
    -re ".*assword:.*" {
        exp_send "password\r";
        exp_send "exit\r"
    }
    -re "Warning:.*Warning.*Warning.*" { #2: to debug the expect hanging
        exp_send "exit\r"
    }
}

interact;

Here's the error msg (expect is on debug mode):

spawn /usr/bin/ssh remote

parent: waiting for sync byte

parent: telling child to go ahead

parent: now unsynchronized from child

spawn: returns {24373}

Gate keeper glob pattern for '.*assword:.*' is '*assword:*'. Activating booster.
Gate keeper glob pattern for '.*sh.*' is '*sh*'. Activating booster.

expect: does "" (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=no

".*sh.*"? Gate "*sh*"? gate=no

Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.

expect: does "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\n" (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=no

".*sh.*"? Gate "*sh*"? gate=no

root@fe80::cccc:48ff:fe33:3344%en4's password: 

expect: does "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nroot@fe80::cccc:48ff:fe33:3344%en4's password: " (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=yes re=yes

expect: set expect_out(0,string) "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nroot@fe80::cccc:48ff:fe33:3344%en4's password: "

expect: set expect_out(spawn_id) "exp6"

expect: set expect_out(buffer) "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nroot@fe80::cccc:48ff:fe33:3344%en4's password: "

send: sending "password\r" to { exp6 }

expect: continuing expect

expect: does "" (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=no

".*sh.*"? Gate "*sh*"? gate=no

expect: does "\r\n" (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=no

".*sh.*"? Gate "*sh*"? gate=no

expect: does "\r\n\u001b[?1034h" (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=no

".*sh.*"? Gate "*sh*"? gate=no

-sh-3.2# 

expect: does "\r\n\u001b[?1034h-sh-3.2# " (spawn_id exp6) match regular expression ".*assword:.*"? Gate "*assword:*"? gate=no

".*sh.*"? Gate "*sh*"? gate=yes re=yes

expect: set expect_out(0,string) "\r\n\u001b[?1034h-sh-3.2# "

expect: set expect_out(spawn_id) "exp6"

expect: set expect_out(buffer) "\r\n\u001b[?1034h-sh-3.2# "

send: sending "mount -uw /\r" to { exp6 }

send: sending "nvram -some-command" to { exp6 }

send: sending "exit\r" to { exp6 }

interact: received eof from spawn_id exp0

parent: waiting for sync byte

parent: telling child to go ahead

parent: now unsynchronized from child

spawn: returns {24376}

Gate keeper glob pattern for '^id_rsa.pub.*' is 'id_rsa?pub*'. Activating booster.
Gate keeper glob pattern for '.*assword:.*' is '*assword:*'. Activating booster.
Gate keeper glob pattern for 'Warning:.*Warning.*Warning.*' is ''. Not usable, disabling the performance booster.

expect: does "" (spawn_id exp1) match regular expression "^id_rsa.pub.*"? Gate "id_rsa?pub*"? gate=no

".*assword:.*"? Gate "*assword:*"? gate=no

"Warning:.*Warning.*Warning.*"? (No Gate, RE only) gate=yes re=no

expect: does "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\n" (spawn_id exp1) match regular expression "^id_rsa.pub.*"? Gate "id_rsa?pub*"? gate=no

".*assword:.*"? Gate "*assword:*"? gate=no

"Warning:.*Warning.*Warning.*"? (No Gate, RE only) gate=yes re=no

expect: does "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\n" (spawn_id exp1) match regular expression "^id_rsa.pub.*"? Gate "id_rsa?pub*"? gate=no

".*assword:.*"? Gate "*assword:*"? gate=no

"Warning:.*Warning.*Warning.*"? (No Gate, RE only) gate=yes re=no

expect: does "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\n\r\n\r\n\r\n" (spawn_id exp1) match regular expression "^id_rsa.pub.*"? Gate "id_rsa?pub*"? gate=no

".*assword:.*"? Gate "*assword:*"? gate=no

"Warning:.*Warning.*Warning.*"? (No Gate, RE only) gate=yes re=yes

expect: set expect_out(0,string) "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\n\r\n\r\n\r\n"

expect: set expect_out(spawn_id) "exp1"

expect: set expect_out(buffer) "Warning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\nWarning: Permanently added 'fe80::cccc:48ff:fe33:3344%en4' (ECDSA) to the list of known hosts.\r\n\r\n\r\n\r\n\r\n\r\n"

send: sending "exit\r" to { exp1 }

interact: spawn id exp0 not open
    while executing
"interact"
    (file "./Resources/Setup.command" line 44)

ERROR: Build failed: exit status 1

Any help would be appreciated! Thanks in advance BTW- the problem was annoying enough for me to figure out, as this is my very first post to stack overflow. :)

Solution

  • Try like this:

    spawn /usr/bin/ssh remote

expect {
    -re ".*assword:.*" {
        exp_send "password\r";
        exp_continue
    }
    -re ".*-sh-.*" {
        exp_send "mount -uw /; nvram -somecommand; exit\r"
        exp_continue
    }
    eof {}
}
wait

set rsaFile "/var/root/Resources/id_rsa.pub"
spawn /usr/bin/scp $rsaFile remote:/var/root/id_rsa.pub;

expect {
    -re ".*assword:.*" {
        exp_send "password\r";
        exp_continue
    }
    eof {}
}
wait

    Your interact may fail because at that time the spawned process may have already exited and obviously it cannot interact with a dead process.

    For your script, my expect eof basically have the same effect with your interact. They all wait for the spawned process to finish. The difference here is timing. And expect eof just does not generate error when the spawned process exits.