Yandex.Weather parsing

Question

I'm trying to download the 7 days forecast from https://www.yandex.com/weather/moscow The problem is all the days except today have the same class. How do I get the forecast for 7 days (or at least for 9)?

I'm trying the BeautifulSoap library. I've got today weather, but all the other days are a problem.

Here is the code I have:

import urllib.request
from bs4 import BeautifulSoup

def get_html(url):
    response = urllib.request.urlopen(url)
    return response.read()

def parse_today(html):
    soup = BeautifulSoup(html, "html.parser")
    temp = soup.find('div', class_='temp fact__temp fact__temp_size_s').get_text().encode('utf-8').decode('utf-8', 'ignore')
    return temp

def parse_next_day(day_num, html):
    # ?????
    pass

def main():
    temp = parse_today(get_html('https://yandex.ru/weather/moscow'))
    print("Now the temperature is: ", temp)
    for i in range(1,6):
        next_temp = parse_next_day(i+1, get_html('https://yandex.ru/weather/moscow'))
        print("The day", i+1, "temperature is : ", next_temp)

if __name__ == '__main__':
    main()

Answer 1

Data is dynamically pulled from an url you can find in the network tab. It returns html. You can isolate the day blocks for forecast using css selector .card:not(.adv). Uses bs4 4.7.1 +. Example of parsing out feels like temps:

import requests
from bs4 import BeautifulSoup as bs

r = requests.get('https://www.yandex.com/weather/segment/details?offset=0&lat=55.753215&lon=37.622504&geoid=213&limit=10', headers = {'User-Agent':'Mozilla/5.0'})
soup = bs(r.content, 'lxml')

for card in soup.select('.card:not(.adv)'):
    date = ' '.join([i.text for i in card.select('[class$=number],[class$=month]')])
    print(date)
    temps = list(zip(
                      [i.text for i in card.select('.weather-table__daypart')]
                    , [i.text for i in card.select('.weather-table__body-cell_type_feels-like .temp__value')]
                ))
    print(temps)