Search code examples
expect

getting log from telnet session and writing relevant info to file but line gets overwritten by timestamp variable in expect script



Here is a sample remote session that I am automating

[sample-session]

$telnet 172.16.11.2
Trying 172.16.11.2...
Connected to 172.16.11.2.
Escape character is '^]'.
BCM96858 Broadband Router
Login: user
Password: 
 > voice show
    

Global Parameters:
------------------
BoundIfName                : bronu8.2501
IP address family          : IPv4
BoundIpAddr                : 172.16.11.2
Voice Log Levels           : general=3,cmgr=3,disp=3,sipcctk=3,bos=3,ept=3,cms=3,prov=3,lhapi=3,istw=3,dsphal=3,slicslac=3
Management Protocol        : OMCI
Voice DNS Server Enable    : Off
Voice DNS Server Primary   : 0.0.0.0
Voice DNS Server Secondary : 0.0.0.0

Service Provider 0:
--------------------
   Associated Voice Profile: 1
   Locale                  : IND
   DTMFMethod              : InBand
   HookFlashMethod         : None
   DigitMap                : x+T
   Log Server Addr         : 0.0.0.0
   Log Server Port         : 0
   T38                     : on
   V18                     : on
   RTPDSCPMark             : 46
   European flash          : Off
   SIP:                       
      SIP mode             : RFC3261
      Domain               : 
      Port                 : 5060
      Transport            : UDP
      SIP URI for TLS calls: Off
      RegExpires           : 300
      RegRetryInterval     : 60
      DSCPMark             : 46
      Registrar Addr       : 172.16.11.201
      Registrar Port       : 5060
      Proxy Addr           : 172.16.11.201
      Proxy Port           : 5060
      OutBoundProxy Addr   : 172.16.11.201
      OutBoundProxy Port   : 5060
      Music Server Addr    : 0.0.0.0
      Music Server Port    : 0
      Conferencing URI     : 
      Conferencing Option  : Local
      Failover Enable      : Off
      Sip OPTIONS Enable   : Off
      Back-to-primary cfg  : Disabled
      Secondary Domain              : 
      Secondary Registrar Addr      : 0.0.0.0
      Secondary Registrar Port      : 5060
      Secondary Proxy Addr          : 0.0.0.0
      Secondary Proxy Port          : 5060
      Secondary Outbound Proxy Addr : 0.0.0.0
      Secondary Outbound Proxy Port : 5060
      To Tag Matching      : On
      Timer B ( in ms )    : 32000
      Timer F ( in ms )    : 32000
      SRTP Usage Option    : Disabled

   Account 0:
   -----------
      ActivationStatus        : Enabled
      VoipServiceStatus       : Up
      CallStatus              : Idle
      Associated LineInst     : 1
      PhysEndpt               : 0
      Extension               : 122
      DisplayName             : 
      AuthName                : 122
      AuthPwd                 : 0000
      TxGain                  : 0 dB
      RxGain                  : 0 dB
      CALLFEATURES:               
         MWI                  : off
         Caller ID number     : on
         Caller ID name       : on
         CallWaiting          : on
         CFWDNum              : 
         CallFwdAll           : off
         CallFwdBusy          : off
         CallFwdNoans         : off
         AnonymousOutgoingCall: off
         AnonymousCallRcvBlock: off
         DoNotDisturb         : off
         CallCompOnBusy       : off
         SpeedDial            : off
         WarmLine             : off
         WarmLineNum          : 
         CallBarring          : off
         CallBarringMode      : None
         CallBarringPin       : 9999
         CallBarringDigitMap  : 
         NetPrivacy           : on
      CODECSETTINGS:           
         VAD                  : on
         pTime                : 20
         CodecList            : (0) G.711MuLaw
                                (1) G.711ALaw
                                (2) G.723.1
                                (3) G.726_16
                                (4) G.726_24
                                (5) G.726_32

   Account 1:
   -----------
      ActivationStatus        : Enabled
      VoipServiceStatus       : Up
      CallStatus              : Idle
      Associated LineInst     : 2
      PhysEndpt               : 1
      Extension               : 121
      DisplayName             : 
      AuthName                : 121
      AuthPwd                 : 0000
      TxGain                  : 0 dB
      RxGain                  : 0 dB
      CALLFEATURES:               
         MWI                  : off
         Caller ID number     : on
         Caller ID name       : on
         CallWaiting          : on
         CFWDNum              : 
         CallFwdAll           : off
         CallFwdBusy          : off
         CallFwdNoans         : off
         AnonymousOutgoingCall: off
         AnonymousCallRcvBlock: off
         DoNotDisturb         : off
         CallCompOnBusy       : off
         SpeedDial            : off
         WarmLine             : off
         WarmLineNum          : 
         CallBarring          : off
         CallBarringMode      : None
         CallBarringPin       : 9999
         CallBarringDigitMap  : 
         NetPrivacy           : on
      CODECSETTINGS:           
         VAD                  : on
         pTime                : 20
         CodecList            : (0) G.711MuLaw
                                (1) G.711ALaw
                                (2) G.723.1
                                (3) G.726_16
                                (4) G.726_24
                                (5) G.726_32
 > exit

Bye bye. Have a nice day!!!
Connection closed by foreign host.

[/sample-session]

My aim is to obtain lines containing ActivationStatus and VoipServiceStatus.

So I wrote this script

#!/usr/bin/expect -f
set timeout 2
set ip_addr [lindex $argv 0]
set OUTFILE "voip_status.log"
spawn telnet 172.16.11.$ip_addr
expect {
  timeout { send_user "\nFailed to get login prompt for $ip_addr\n"; exit 1 }
  eof { send_user "\ntelnet failure for server $ip_addr\n"; exit 2 }
  "*Login:"
}
        #expect "*Login:"
        send -- "user\r" 
        expect "*Password:"
        send -- "user\r" 
        expect ">"
        send -- "voice show\r"
        expect ">"
        set voice_full_output $expect_out(buffer)
        set voice_stat_lines [ split $voice_full_output "\n" ]
        foreach line $voice_stat_lines {
            if {[string match "*VoipServiceStatus*" $line]} {
                set timestamp [timestamp -format %Y-%m-%d_%H:%M:%S]
                set chan [open $OUTFILE a+]
                    #puts "ok - MATCH $line"
                    puts $chan "$line $timestamp"
                close $chan
                }
            if {[string match "*ActivationStatus*" $line]} {
                set chan [open $OUTFILE a+]
                    #puts "ok - MATCH $line"
                    puts $chan "$line"
                close $chan
                }
        }
        send -- "exit\r"
expect eof

Here is my output

$ cat voip_status.log
      ActivationStatus        : Enabled
 2021-03-17_11:32:35tus       : Up
      ActivationStatus        : Enabled
 2021-03-17_11:32:35tus       : Up

As you can see that time stamp has overwritten the line I had filtered out.
I wanted timestamp to appear at the end of line .
Why is this happening ?

Here I made an experimental setup which anybody can run to see this behaviour.
Contents of 'output-voice-log-for-experiment.sh' follow
[file to give input]

#!/usr/bin/perl
print "
    
       Account 0:
   -----------
      ActivationStatus        : Enabled
      VoipServiceStatus       : Up
      CallStatus              : Idle
      Associated LineInst     : 1
      PhysEndpt               : 0
      Extension               : 122
      DisplayName             : 
      AuthName                : 122
      AuthPwd                 : 0000
      TxGain                  : 0 dB
      RxGain                  : 0 dB
      CALLFEATURES:               
         MWI                  : off
         Caller ID number     : on
         Caller ID name       : on
         CallWaiting          : on
         CFWDNum              : 
         CallFwdAll           : off
         CallFwdBusy          : off
         CallFwdNoans         : off
         AnonymousOutgoingCall: off
         AnonymousCallRcvBlock: off
         DoNotDisturb         : off
         CallCompOnBusy       : off
         SpeedDial            : off
         WarmLine             : off
         WarmLineNum          : 
         CallBarring          : off
         CallBarringMode      : None
         CallBarringPin       : 9999
         CallBarringDigitMap  : 
         NetPrivacy           : on
      CODECSETTINGS:           
         VAD                  : on
         pTime                : 20
         CodecList            : (0) G.711MuLaw
                                (1) G.711ALaw
                                (2) G.723.1
                                (3) G.726_16
                                (4) G.726_24
                                (5) G.726_32

   Account 1:
   -----------
      ActivationStatus        : Enabled
      VoipServiceStatus       : Up
      CallStatus              : Idle
      Associated LineInst     : 2
      PhysEndpt               : 1
      Extension               : 121
      DisplayName             : 
      AuthName                : 121
      AuthPwd                 : 0000
      TxGain                  : 0 dB
      RxGain                  : 0 dB
      CALLFEATURES:               
         MWI                  : off
         Caller ID number     : on
         Caller ID name       : on
         CallWaiting          : on
         CFWDNum              : 
         CallFwdAll           : off
         CallFwdBusy          : off
         CallFwdNoans         : off
         AnonymousOutgoingCall: off
         AnonymousCallRcvBlock: off
         DoNotDisturb         : off
         CallCompOnBusy       : off
         SpeedDial            : off
         WarmLine             : off
         WarmLineNum          : 
         CallBarring          : off
         CallBarringMode      : None
         CallBarringPin       : 9999
         CallBarringDigitMap  : 
         NetPrivacy           : on
      CODECSETTINGS:           
         VAD                  : on
         pTime                : 20
         CodecList            : (0) G.711MuLaw
                                (1) G.711ALaw
                                (2) G.723.1
                                (3) G.726_16
                                (4) G.726_24
                                (5) G.726_32
 > exit
"

[/file to give input]
Contents of 'process-experimental-echo.sh' follow
[file to process input]

#!/usr/bin/expect -f
spawn ./output-voice-log-for-experiment.sh 
    set timeout 20
    set OUTFILE "experimental_voip_status.log"
            expect ">"
            set voice_full_output $expect_out(buffer)
            set voice_stat_lines [ split $voice_full_output "\n" ]
            foreach line $voice_stat_lines {
                if {[string match "*VoipServiceStatus*" $line]} {
                    set timestamp [timestamp -format %Y-%m-%d_%H:%M:%S]
                    set chan [open $OUTFILE a+]
                        puts "ok - MATCH $line"
                        puts $chan "$line $timestamp"
                    close $chan
                    }
                if {[string match "*ActivationStatus*" $line]} {
                    set chan [open $OUTFILE a+]
                        puts "ok - MATCH $line"
                        puts $chan "$line"
                    close $chan
                    }
            }

[/file to process input]
Now simply run ./process-experimental-echo.sh and see this behaviour yourself.
Here is the output file you will get

$ cat experimental_voip_status.log
      ActivationStatus        : Enabled
 2021-03-17_12:27:11tus       : Up
      ActivationStatus        : Enabled
 2021-03-17_12:27:11tus       : Up



Thanks.


Solution

  • If you look at the output file using cat -vet it shows the non-printing control characters, tabs, and end-of-line, and you get:

         ActivationStatus        : Enabled^M$
         VoipServiceStatus       : Up^M 2021-03-17_09:02:53$
    

    You can see ^M in the middle of a line, which is carriage-return. On a terminal this makes the cursor go back to the start of the line, and start overprinting.

    This is the typical result of spawn, which runs a command through a pty, which alters each output newlines into carriage-return and newline. This is normal and desired for a pty. You can either turn it off (see stty -onlcr), or knowing that it is likely to be there, you can remove it in your script: after the foreach ... line, trim the variable $line to remove \r:

    set line [string trimright $line "\r"]