Search code examples
pythonbeautifulsoupjupyter-notebookgmail

How do I import gmail notes into python?


from bs4 import BeautifulSoup

soup = BeautifulSoup(open("https://mail.google.com/mail/u/0/#label/Notes"), features="lxml")

print(soup.prettify())

The above doesn't work for me. I am not sure how to do authentication and what web address to use inside open command.


Solution

  • So Google varies its login procedures from time to time to ensure security and prevent bot logins. Which means there is code to login using BeautifulSoup but it no longer works.

    The reason that 2016, and earlier, code will not work is that Gmail has added an encryption using JS as part of its authentication, captchas, and other measures which would have to be reverse engineered in order to use BSoup. Basically, removing the ability to just "scrape" the data.

    If you want to download the emails for processing, it is possible using the Gmail API or the Google Cloud API but this is not always an option and would be beyond the scope of this question.