I try to run 3rd party process (nsqd.exe) from my python script but when I do nsqd fails to bind socket. I've no idea why.
The script I'm using:
import subprocess
import sys
proc = subprocess.Popen(['nsqd.exe', '-tcp-address="127.0.0.1:{}"'.format(sys.argv[1]),
'-http-address="127.0.0.1:{}"'.format(sys.argv[2])])
print("the commandline is {}".format(proc.args))
proc.wait()
sys.exit(proc.returncode)
And the output:
D:\bsm.tar\bsm\final\nsqd>python nsqd.py 4150 4151
the commandline is ['nsqd.exe', '-tcp-address="127.0.0.1:4150"', '-http-address="127.0.0.1:4151"']
[nsqd] 2016/09/26 21:41:51.974681 nsqd v0.3.8 (built w/go1.6.2)
[nsqd] 2016/09/26 21:41:51.975681 ID: 864
[nsqd] 2016/09/26 21:41:51.979675 NSQ: persisting topic/channel metadata to nsqd.864.dat
[nsqd] 2016/09/26 21:41:52.004711 FATAL: listen ("127.0.0.1:4150") failed - listen tcp: lookup "127.0.0.1: getaddrinfow: No such host is known.
If I run that by myself directly everything works fine:
D:\bsm.tar\bsm\final\nsqd>nsqd.exe -tcp-address="127.0.0.1:4150" -http-address="127.0.0.1:4151"
[nsqd] 2016/09/26 21:42:20.093848 nsqd v0.3.8 (built w/go1.6.2)
[nsqd] 2016/09/26 21:42:20.094850 ID: 864
[nsqd] 2016/09/26 21:42:20.095851 NSQ: persisting topic/channel metadata to nsqd.864.dat
[nsqd] 2016/09/26 21:42:20.127984 TCP: listening on 127.0.0.1:4150
[nsqd] 2016/09/26 21:42:20.127984 HTTP: listening on 127.0.0.1:4151
[nsqd] 2016/09/26 21:42:22.111580 NSQ: persisting topic/channel metadata to nsqd.864.dat
[nsqd] 2016/09/26 21:42:22.111580 TCP: closing 127.0.0.1:4150
[nsqd] 2016/09/26 21:42:22.112553 HTTP: closing 127.0.0.1:4151
[nsqd] 2016/09/26 21:42:22.135635 NSQ: closing topics
[nsqd] 2016/09/26 21:42:22.135635 QUEUESCAN: closing
[nsqd] 2016/09/26 21:42:22.135635 LOOKUP: closing
[nsqd] 2016/09/26 21:42:22.135635 ID: closing
D:\bsm.tar\bsm\final\nsqd>
Maybe somebody have idea what is wrong?
Win10, python352. Running as admin does not help.
Thanks.
Remove the double quotes in your Popen so it becomes:
proc = subprocess.Popen(['nsqd.exe',
'-tcp-address=127.0.0.1:{}'.format(sys.argv[1]),
'-http-address=127.0.0.1:{}'.format(sys.argv[2])
])
Before passing your command to CreateProcess, Python converts the list to a string using subprocess.list2cmdline:
>>> subprocess.list2cmdline(['nsqd.exe', '-tcp-address="127.0.0.1:4150"', '-http-address="127.0.0.1:4151"'])
'nsqd.exe -tcp-address=\\"127.0.0.1:1234\\" -http-address=\\"127.0.0.1:1234\\"
nsqd.exe thinks "127.0.0.1 is the hostname - hence the failed lookup.
The reason the double quotes work on the command line is that they have special meaning when a function such as CommandLineToArgvW is used to split the command line into individual arguments: normally arguments are delimited by whitespace, but when a quoted string is encountered, the quotes are stripped and the entire string becomes one argument.
This is also why Python is \-escaping the quotes: it expects the resultant line to be parsed in the above manner.
If you pass Popen a string rather than a list, list2cmdline will not be called and you should get the same results as removing the double quotes (i.e. it will be like running it from the command line):
proc = subprocess.Popen('nsqd.exe "-tcp-address=127.0.0.1:{}" '
'"-http-address=127.0.0.1:{}"'
.format(sys.argv[1], sys.argv[2]))
You can see this illustrated in the following (perhaps contrived) example:
import subprocess
subprocess.Popen('c:\python27\python.exe "--version"')
subprocess.Popen(['c:\python27\python.exe', '"--version"'])
The first Popen will print the python version. The second will look for a file named "--version": can't open file '"--version"': [Errno 22] Invalid argument