Search code examples
pythonweb-scrapingpycharm

Make PyCharm successfully run session.get() from requests_html


I am trying to web scrape with python in PyCharm using requests_html. The following code doesn't return an error in the terminal using the same venv as my PyCharm project (python 3.8.10), but when I use PyCharm I get an error.

The code:

from requests_html import HTMLSession
session = HTMLSession
response = session.get('https://python.org')

After running response = session.get('https://python.org') in PyCharm I get this error:

Traceback (most recent call last):
  File "/usr/lib/python3.8/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 11, in <module>
TypeError: get() missing 1 required positional argument: 'url'

How can I make PyCharm successfully run session.get()?


Solution

  • It looks like you accidentally left out the parentheses here:

    session = HTMLSession

    So: session.get --> <function Session.get at 0x10377e550>.

    help(session.get) --> get(self, url, **kwargs).

    It's trying to call the get() method on a class HTMLSession rather than an instance of the class, that is, a session. In that case, the function receives the string argument in place of the implicit self argument and it doesn't receive the url argument.

    Whereas after:

    session = HTMLSession()

    session.get --> <bound method Session.get of <requests_html.HTMLSession object at 0x10276c460>>

    help(session.get) --> get(url, **kwargs).

    session.get('https://python.org') --> <Response [200]>.