How to suppress gcloud connection closed warning when executed from python

Question

def instance_list():
instances_list = "gcloud compute instances list --format='json(name, status, zone)' --filter='(status=running)' --project reference-ether-853"
instance_output_json = json.loads(subprocess.check_output(shlex.split(instances_list)))
# instance_output_table = "gcloud compute instances list --format='table(name, status, zone)' --filter='(status=running)' --project reference-ether-853"
# result = subprocess.check_output(shlex.split(instance_output_table))
for instance_row in instance_output_json:
    instance_name = instance_row['name']
    instance_status = instance_row['status']
    instance_zone = instance_row['zone'].rsplit("/")[-1]
    if instance_name.startswith('ci-') and instance_status == 'RUNNING':
        uptime = "gcloud compute ssh --zone {} {} --ssh-flag='-t' -- command 'uptime'".format(instance_zone,
                                                                                              instance_name)
        instances_up = subprocess.check_output(shlex.split(uptime))
        print(instance_name, instance_zone)

While executing the Output is returned as

instanceName us-east1-b Connection to 31.196.126.66 closed.

I want to suppress "Connection to 31.196.126.66 closed." and print only the instance name and zone.

Answer 1

I think you can't do this without something like the following:

gcloud compute ssh --zone {} {} --ssh-flag='-t' --command 'uptime' 2>&1 \
| head --lines=1

gcloud compute ssh is a wrapper around your machine's ssh command and so you must resorting to parsing stdout|stderr.

I've not tried this using Python's subprocess but you should manage the pipe explicitly (i.e. you'll need two commands and subprocess.PIPE)

It is generally inelegant to script shell commands from a programming language because, as you're experiencing, it's difficult to 'type' the output in order to e.g. parse it, process errors.

It is generally much better to use one of the Google-provided libraries (available for every service in multiple languages) to interact with services such as Compute Engine from Python.

See:

https://cloud.google.com/compute/docs/tutorials/python-guide

However (:-)), in this case, the alternative is rather challenging.

IIUC, you cannot grab an uptime measurement for an instance from the Compute Engine API but would need to use a Monitoring Metric (either uptime or uptime_total, I'm unsure of the difference).

I encourage you to consider replacing the gcloud compute instances list shell command with the equivalent API method. See the example on Google APIs Explorer:

https://cloud.google.com/compute/docs/reference/rest/v1/instances/list#examples