Want to run a curl command either through python's subprocess or pycurl

Question

I have the following command which adds a user to the administrator group of a gerrit instance,

 curl -X POST -H "Content-Type:application/json;charset=UTF-8" -u nidhi:pswd http://host_ip:port/a/groups/Administrators/members.add -d '{"members":["user@example.com"]}'

When I run this command on my terminal, it runs perfectly and gives the expected output.

But, I want to execute this command in python either using the subprocess or pycurl library.

Using subprocess I wrote the following code,

def add_user_to_administrator(u_name,url):
    bashCommand = 'curl -X POST -H "Content-Type:application/json;charset=UTF-8" -u nidhi:pswd http://'+url+'/a/groups/Administrators/members.add -d '+"'"+'{"members":["$u_n@example.com"]}'+"'"
    bashCommand = string.Template(bashCommand).substitute({'u_n':u_name})
    print bashCommand.split()
    process = subprocess.Popen(bashCommand.split())

It shows no error but no changes are seen in the administrator group.

I tried the same using pycurl,

def add_user_to_administrator2(u_name,url):
    pf = json.dumps({"members":[str(str(u_name)+"@example.com")]})
    headers = ['Content-Type:application/json;charset=UTF-8']
    pageContents = StringIO.StringIO()


    p = pycurl.Curl()
    p.setopt(pycurl.FOLLOWLOCATION, 1)
    p.setopt(pycurl.POST, 1)
    p.setopt(pycurl.HTTPHEADER, headers)
    p.setopt(pycurl.POSTFIELDS, pf)
    p.setopt(pycurl.WRITEFUNCTION, pageContents.write)
    p.setopt(pycurl.VERBOSE, True)
    p.setopt(pycurl.DEBUGFUNCTION, test)
    p.setopt(pycurl.USERPWD, "nidhi:pswd")
    pass_url=str("http://"+url+"/a/groups/Administrators/Administrators/members.add").rstrip('\n')
    print pass_url
    p.setopt(pycurl.URL, pass_url)
    p.perform()

    p.close() 
    pageContents.seek(0)
    print pageContents.readlines()

This throws an error, it cannot find the account members.

The variable mentioned url is of the form host_ip:port.

I have tried a lot to fix these errors. I dont know where I am going wrong. Any help would be appreciated.

Answer 1

a) string escaping

For the subprocess/curl usage, you should be escaping your string tokens rather than manually adding extra ':

...stuff'+"'"+'more.stuff...

Escape using \ before the character i.e. using

"curl -X POST -H \"Content-Type:application/json;charset=UTF-8\""

will keep the " around the Content-Type section.

More on escape characters here: Lexical Analysis - String Literals

...The backslash () character is used to escape characters that otherwise have a special meaning...

b) popen

Looking at the popen docs their example uses shlex.split() to split their command line into args. shlex splits the string a bit differently:

print(bashCommand.split())
['curl', '-X', 'POST', '-H', '"Content-Type:application/json;charset=UTF-8"', '-u', 'nidhi:pswd', 'http://TEST_URL/a/groups/Administrators/members.add', '-d', '\'{"members":["TEST_USER@example.com"]}\'']

print(shlex.split(bashCommand))
['curl', '-X', 'POST', '-H', 'Content-Type:application/json;charset=UTF-8', '-u', 'nidhi:pswd', 'http://TEST_URL/a/groups/Administrators/members.add', '-d', '{"members":["TEST_USER@example.com"]}']

you can see shlex removes excess quoting.

c) http response code

Try using -I option in curl to get a HTTP response code back (and the rest of the HTTP headers):

$curl -h
...
-I, --head          Show document info only

Even though you're using subprocess to start/make the request, it should still print the return value to the console(stdout).

d) putting it all together

I changed the how the url and u_name are interpolated into the string.

import shlex
import subprocess


def add_user_to_administrator(u_name, url):
    bashCommand = "curl -I -X POST -H \"Content-Type:application/json;charset=UTF-8\" -u nidhi:pswd http://%(url)s/a/groups/Administrators/members.add -d '{\"members\":[\"%(u_n)s@example.com\"]}'"
    bashCommand = bashCommand % {'u_n': u_name, 'url': url}
    args = shlex.split(bashCommand)
    process = subprocess.Popen(args)

add_user_to_administrator('TEST_USER', 'TEST_URL')

If none of this helps, and you're getting no response from gerrit, I'd check gerrit logs to see what happens when it receives your request.