Search code examples
pythonweb-scrapingbeautifulsouppython-requests

How to get text from a div or class via requests


I am writing a script that locates and prints the transaction amount of the most recent transaction at this website https://ltc.tokenview.io/en/address/M8T1B2Z97gVdvmfkQcAtYbEepune1tzGua

This is the code I am trying to use.

def getRecentTransaction(address):
    r = requests.get(f'https://ltc.tokenview.io/en/address/{address}')
    time.sleep(5)
    soup = BeautifulSoup(r.text, 'html.parser')
    amount = soup.find(class_="input_value")
    amountStuff = amount.text.strip()
    print(amountStuff)

However this is just returning "none". Now I am aware that there are multiple "input-value" classes on the website but when I did find all I got "[]". The only way I was able to retrieve the text was using selenium's by Xpath function. However, I am trying to write this so people can just input an address and then get the recent transaction amount. The Xpath method is too specific.

Any help is appreciated!


Solution

  • The data you see on the page is loaded with help of javascript, so BeautifulSoup doesn't see it. To get the data you can do:

    import requests
    
    api_url = "https://ltc.tokenview.io/api/address/balancetrend/ltc/M8T1B2Z97gVdvmfkQcAtYbEepune1tzGua"
    
    data = requests.get(api_url).json()
    
    # print most recent:
    print(data["data"][0])
    

    Prints:

    {'2024-01-06': '2504667.37296058'}
    

    The most recent transaction:

    import re
    
    import requests
    
    url = "https://ltc.tokenview.io/en/address/M8T1B2Z97gVdvmfkQcAtYbEepune1tzGua"
    html_text = requests.get(url).text
    
    inp, out = re.search(r'value:"([^"]+).*?value:"([^"]+)', html_text).groups()
    print(f"{inp=} {out=}")
    

    Prints:

    inp='0.02387814' out='0.02319739'