How do I scrape a whole table when they all use the same class?, I can so far get the names only

Question

I'm trying to get the data in the table labelled "Key rates". However, I can only extract the Names as they have a unique style ( ).

I want data from the table and arranged into a table by using Pandas. Please help

Code:

URL = "https://www.centralbank.go.ke/"
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')

job_elems = soup.find_all('td', class_="tg-4eph")             
for job_elem in job_elems:                                                      
    title_elem = job_elem.find('small')                                             
    if title_elem:                                                              
        print(title_elem.text.strip())

Answer 1

One way to approach this would be to look for a table that has the string "Key Rates" associated with a th element.

You may want to use column names other than those derived from the HTML

Something like this:

import requests
from bs4 import BeautifulSoup as BS
from bs4.element import Tag
import pandas as pd
import io

TH = "Key Rates"

def is_table(table: Tag, text: str=TH) -> bool:
    for th in table.select("tr th"):
        if th.get_text() == text:
            return True
    return False

with requests.get("https://www.centralbank.go.ke") as response:
    response.raise_for_status()
    for table in BS(response.text, "lxml").select("table.tg"):
        if is_table(table):
            html = io.StringIO(str(table))
            df = pd.read_html(html)[0].drop(columns=[TH])
            mapper = {k: v for k, v in zip(df.columns, ["Rate type", "Percentage", "Date"])}
            print(df.rename(columns=mapper).to_string(index=False))
            break

Output:

          Rate type Percentage       Date
  Central Bank Rate     13.00% 05/06/2024
    Inter-Bank Rate     13.26% 04/07/2024
CBK Discount Window     16.00% 05/06/2024
      91-Day T-Bill    15.977% 01/07/2024
               REPO      0.00% 08/05/2024
     Inflation Rate      4.64%  June,2024
       Lending Rate     16.45% April,2024
       Savings Rate      4.14% April,2024
       Deposit Rate     10.77% April,2024
               KBRR       8.9% 27/06/2016