Web Scraping for getting data with Python NoneType ERROR

Question

I'm trying to get dollar, prices for my school project. So I decide to use web scraping for this but I have a problem about it. When I try to use my code on server it gives me NoneType ERROR. It works on google colab but I can't use on my pc or server. How can I solve this guys?

Web Scrape code ;

def dolar():
  headers = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15'
  url = f'https://finance.yahoo.com/quote/TRY=X/'
  r = requests.get(url)
  soup = bs(r.text, 'html.parser')
  dolar = soup.find("div", {"class": "container yf-mgkamr"}).find_all("span")[0].text
  return dolar

EROOR ;

Traceback (most recent call last):
  File "/Users/user/Desktop/API/main.py", line 38, in <module>
    dolar()
  File "/Users/user/Desktop/API/main.py", line 35, in dolar
    dolar = soup.find("div", {"class": "container yf-mgkamr"}).find_all("span")[0].text
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find_all'
(.venv) user@192 API % 

I tried to change my main website, tried to use without ".find_all" method. It doesn't change anything.

Answer 1

You should probably use this

import requests
import time


def dolar():
    now = time.time() - 10
    headers = {
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15"
    }
    url = f"https://query1.finance.yahoo.com/v8/finance/chart/TRY=X?period1={int(now)}&period2={int(now)}"
    response = requests.get(url, headers=headers)
    return response.json()['chart']['result'][0]['meta']['regularMarketPrice']


print(dolar())