Search code examples
pythonpython-requestsgrequests

How to send multiple HTTP requests using Python

I'm trying to create a script that checks all possible top level domain combinations based on a given root domain name.

What I've done is generate a list of all possible TLD combinations and what I want to do is make multiple HTTP requests, analyze its results and determine whether a root domain name has multiple active top level domains or not.

Example: A client of mine has this active domains:

  • domain.com
  • domain.com.ar
  • domain.ar

I've tried using grequests but get this error:

TypeError: 'AsyncRequest' object is not iterable

Code:

import grequests


responses = grequests.get([url for url in urls])
grequests.map(responses)
Solution

  • As I mentioned, you cannot put code as a parameter. What you want is to add a list and you can using an inline for loop, like this: [url for url in urls]

    It is called list comprehensions and more information about this can be found over here: https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

    So I just added two brackets:

    import grequests

responses = (grequests.get(u) for u in urls)
grequests.map(responses)