Search code examples
pythonhttpasynchronoustwistedtreq

How do I get the response text from a treq request?


I am trying to get started with some example code of the treq library, to little avail. While it is easy to get the status code and a few other properties of the response to the request, getting the actual text of the response is a little more difficult. The print_response function available in this example code is not present in the version that I have:

from twisted.internet.task import react
from _utils import print_response

import treq


def main(reactor, *args):
    d = treq.get('http://httpbin.org/get')
    d.addCallback(print_response)
    return d

react(main, [])

Here is the traceback:

Traceback (most recent call last):
  File "test.py", line 2, in <module>
    from _utils import print_response
ModuleNotFoundError: No module named '_utils'

I am not really sure where to go from here...any help would be greatly appreciated.


Solution

  • Now that I look at it, that example is extremely bad, especially if you're new to twisted. Please give this a try:

    import treq
    from twisted.internet import defer, task
    
    def main(reactor):
        d = treq.get('https://httpbin.org/get')
        d.addCallback(print_results)
        return d
    
    @defer.inlineCallbacks
    def print_results(response):
        content = yield response.content()
        text = yield response.text()
        json = yield response.json()
    
        print(content)  # raw content in bytes
        print(text)     # encoded text
        print(json)     # JSON
    
    task.react(main)
    

    The only thing you really have to know is that the .content(),.text(),.json() return Deferred objects that eventually return the body of the response. For this reason, you need to yield or execute callbacks.

    Let's say you only want the text content, you could this:

    def main(reactor):
        d = treq.get('https://httpbin.org/get')
        d.addCallback(treq.text_content)
        d.addCallback(print)    # replace print with your own callback function
        return d
    

    The treq.content() line of functions make it easy to return only the content, if thats all you care about, and do stuff with it.