Search code examples
pythonpandasweb-scrapingbeautifulsoup

scraping table without table class


im trying to scrape table from https://fxtop.com/en/historical-exchange-rates.php?A=1&C1=USD&C2=IDR&MA=1&DD1=&MM1=08&YYYY1=1995&B=1&P=&I=1&DD2=23&MM2=07&YYYY2=2020&btnOK=Go%21 but im not able to scrape data because i cant find the table class,can anyone help with the right indenfication? thank you in advance.

    import pandas as pd
import requests
from bs4 import BeautifulSoup

    url = "https://fxtop.com/en/historical-exchange-rates.php?A=1&C1=USD&C2=IDR&MA=1&DD1=&MM1=08&YYYY1=1995&B=1&P=&I=1&DD2=23&MM2=07&YYYY2=2020&btnOK=Go%21"

    r = requests.get(url)
html = r.text

    soup = BeautifulSoup(html)
table = soup.find('table', border = 1)
print(table)

    result = pd.DataFrame(data, columns=['month', 'average USD/idr=', 'Min USD/IDR=', 'Max USD/IDR=', 'Nb of working days'])

result.to_csv("usd_.csv", index=False)

df = pd.read_csv("usd_.csv")
pd.set_option('display.max_rows', df.shape[0]+1)
print(df)

Solution

  • import pandas as pd
    
    
    df = pd.read_html(
        "https://fxtop.com/en/historical-exchange-rates.php?A=1&C1=USD&C2=IDR&MA=1&DD1=&MM1=08&YYYY1=1995&B=1&P=&I=1&DD2=23&MM2=07&YYYY2=2020&btnOK=Go%21", header=0)[-3]
    
    print(df)
    df.to_csv("data.csv", index=False)
    

    Output: check online

    enter image description here