Get CSRF token using python requests

I am currently using Python Requests, and need a CSRF token for logging in to a site. from my understanding requests.Session() gets the cookie, but obviously I need the token. And Also I would like to know where to place it in my code. import requests

user_name = input('Username:')
payload = {
'username': 'user_name',
'password': 'randompass123'
}


with requests.Session() as s:
p = s.post('https://examplenotarealpage.com', data=payload)

Solution

See the following code example. You can use it directly to login into a website that only uses cookies to store login information.

import requests

LOGIN_URL = 'https://examplenotarealpage.com'
headers = {
    'accept': 'text/html,application/xhtml+xml,application/xml',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

response = requests.get(LOGIN_URL, headers=headers, verify=False)

headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])
headers['content-type'] = 'application/x-www-form-urlencoded'
payload = {
    'username': 'user_name',
    'password': 'randompass123'
}

response = requests.post(LOGIN_URL, data=payload, headers=headers, verify=False)
headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])

There are a few possible locations of the CSRF token. Different websites use different ways to pass it to browser. Here are some of them:

It can come with response headers, in that case getting it is easy.
Sometimes page meta holds the CSRF token. You have to parse the html content of the page to get it. Find the proper CSS selector for it. See an example:
```
from bs4 import BeautifulSoup
soup = BeautifulSoup(response.text, 'lxml')
csrf_token = soup.select_one('meta[name="csrf-token"]')['content']
```
It can be inside of a script tag with JavaScript code. Getting it will be tricky. But, you can always use regex to isolate it.