Python telnetlib to connect to Scrapy Telnet to read stats

Question

I have a Scrapy spider running for days. I usually try to check stats as how many items it has scraped and so. I simply run following cmds in terminal and it gives me stats.

$ telnet [IP] [PORT]
>>> spider.name
alf-spider
>>> stats.get_stats()
...

Now I want to do this with Python using telnetlib but I can't achieve above results. Following is my python code.

#!/usr/bin/env python

import sys
import getpass
import telnetlib

HOST = "192.168.1.5"

def main():
    ports = ['6023']
    if len(sys.argv) > 1:
        ports = sys.argv[1].split(',')

    for port in ports:
        get_stats(port)

def get_stats(port):
    tn = telnetlib.Telnet(HOST, port)
    tn.write("spider.name\n")
    print tn.read_all()

if __name__ == '__main__':
    main()

Above code if ran just hangs until force closed. What am I missing?

Answer 1

This will work.

#!/usr/bin/env python

import sys
import getpass
import telnetlib

HOST = "192.168.1.5"

def main():
    ports = ['6023']
    if len(sys.argv) > 1:
        ports = sys.argv[1].split(',')

    for port in ports:
        get_stats(port)

def get_stats(port):
    tn = telnetlib.Telnet(HOST, port)
    tn.read_until('>>>')
    tn.write("spider.name\n")
    print tn.read_until('>>>')
    tn.close()

if __name__ == '__main__':
    main()