I am unsure how to extra the 'Settle' column for each state (New South Wales, Victoria, Queensland, South Australia) from this website:
https://www.asxenergy.com.au/futures_au
It seems the numerical data isn't showing.
My starting code is:
from bs4 import BeautifulSoup
from urllib.request import urlopen
url = "https://www.asxenergy.com.au/futures_au"
page = urlopen(url)
html = page.read().decode("utf-8")
soup = BeautifulSoup(html, "html.parser")
print(soup.get_text())
As stated, the data comes from asxenergy.com.au/futures_au/dataset
Use pandas to get the <table> tags. It's not ideally structured, so need a bit of processing here.
import pandas as pd
url = 'https://www.asxenergy.com.au/futures_au/dataset'
dfs = pd.read_html(url)
states = dfs[0].iloc[0].to_list()
table_names = list(dfs[1].columns[0])
data_dfs = dfs[2:]
output = {}
for count, state in enumerate(states):
start_idx = count * 6
end_idx = start_idx + 6
set_idx = list(range(start_idx, end_idx))
temp_table_names = [table_names[i] for i in set_idx]
temp_data_dfs = [data_dfs[i] for i in set_idx]
data_dict = dict(zip(temp_table_names, temp_data_dfs))
output[state] = data_dict
Output:
for state, tables in output.items():
for table_name, df in tables.items():
print(state, '-', table_name)
print(df, '\n')
New South Wales - BMth
BMth Bid Ask Last +/- Vol Settle
0 Jul24 - - - - - 130.75
1 Aug24 - - - - - 117.74
2 Sep24 - - - - - 97.43
3 Oct24 - - - - - 110.00
New South Wales - BQtr
BQtr Bid Ask Last +/- Vol Settle
0 Q324 113.50 115.50 115.50 - 40 115.50
1 Q424 109.50 110.00 110.00 - 16 110.00
2 Q125 126.10 126.70 126.10 - 16 126.10
3 Q225 130.15 130.50 130.15 - 21 130.15
4 Q325 125.00 126.85 126.18 - 23 126.18
New South Wales - BStr
BStr Bid Ask Last +/- Vol Settle
0 FY25 120.29 121.00 - - - 120.38
1 CY25 121.00 122.50 122.00 - 14 122.00
2 FY26 120.00 122.00 121.00 -0.19 4 121.19
3 CY26 120.50 121.25 121.50 +0.69 7 120.81
4 FY27 120.25 121.00 - - - 120.43
New South Wales - Caps
Caps Bid Ask Last +/- Vol Settle
0 Q324 10.00 11.00 10.00 - 17 10.00
1 Q424 20.25 21.50 20.50 +0.25 10 20.25
2 Q125 41.00 43.00 - - - 41.46
3 Q225 25.50 27.00 - - - 26.66
4 Q325 25.00 - - - - 26.54
New South Wales - CapStr
CapStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 24.49
1 CY25 - - - - - 28.96
2 FY26 29.40 - - - - 29.40
3 CY26 28.50 30.00 29.00 -0.35 1 29.35
4 FY27 29.00 30.00 - - - 29.93
New South Wales - PQtr
PQtr Bid Ask Last +/- Vol Settle
0 Q324 116.00 160.00 - - - 160.00
1 Q424 125.00 160.00 - - - 145.00
2 Q125 - - - - - 170.73
3 Q225 - - - - - 150.00
4 Q325 - - - - - 140.00
Victoria - PkStr
PkStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 156.47
1 CY25 - - - - - 144.42
2 FY26 - - - - - 147.97
3 CY26 - - - - - 160.00
4 FY27 - - - - - 148.94
Victoria - BMth
BMth Bid Ask Last +/- Vol Settle
0 Jul24 - - - - - 142.58
1 Aug24 - - - - - 108.21
2 Sep24 - 80.00 - - - 71.28
3 Oct24 - - - - - 58.95
Victoria - BQtr
BQtr Bid Ask Last +/- Vol Settle
0 Q324 107.75 108.50 107.75 - 18 107.75
1 Q424 58.50 58.95 59.00 +0.05 14 58.95
2 Q125 73.40 74.00 73.50 - 13 73.50
3 Q225 93.00 93.80 93.80 - 10 93.80
4 Q325 88.00 89.25 88.50 - 5 88.50
Victoria - BStr
BStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 83.53
1 CY25 76.25 76.85 76.65 +0.29 2 76.36
2 FY26 70.00 73.30 - - - 72.22
3 CY26 65.50 70.24 69.80 +0.25 8 69.55
4 FY27 - - - - - 70.17
Victoria - Caps
Caps Bid Ask Last +/- Vol Settle
0 Q324 6.75 7.5 8.15 +0.65 18 7.5
1 Q424 5.00 7.0 - - - 6.3
2 Q125 27.75 29.0 28.00 - 20 28.0
3 Q225 8.25 11.5 - - - 10.0
4 Q325 10.50 11.5 - - - 10.5
Victoria - CapStr
CapStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 12.88
1 CY25 13.00 15.25 - - - 14.25
2 FY26 13.85 15.25 - - - 14.49
3 CY26 14.50 16.00 - - - 14.81
4 FY27 15.40 - - - - 15.42
Queensland - PQtr
PQtr Bid Ask Last +/- Vol Settle
0 Q324 111.11 - - - - 140.0
1 Q424 - - - - - 64.5
2 Q125 - - - - - 103.0
3 Q225 - - - - - 120.0
4 Q325 - - - - - 119.0
Queensland - PkStr
PkStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 107.07
1 CY25 - - - - - 102.24
2 FY26 - - - - - 77.04
3 CY26 - - - - - 60.00
4 FY27 - - - - - 60.00
Queensland - BMth
BMth Bid Ask Last +/- Vol Settle
0 Jul24 - - - - - 108.07
1 Aug24 - - - - - 89.86
2 Sep24 - - - - - 89.86
3 Oct24 - - - - - 97.00
Queensland - BQtr
BQtr Bid Ask Last +/- Vol Settle
0 Q324 96.00 101.75 - - - 96.00
1 Q424 96.75 97.50 97.00 - 12 97.00
2 Q125 133.00 133.50 133.50 - 18 133.50
3 Q225 99.80 100.45 99.80 - 31 99.80
4 Q325 95.00 96.75 96.01 - 15 96.01
Queensland - BStr
BStr Bid Ask Last +/- Vol Settle
0 FY25 106.50 109.75 - - - 106.45
1 CY25 104.00 104.50 104.00 -0.22 7 104.22
2 FY26 99.50 100.00 99.75 -0.12 4 99.87
3 CY26 97.00 98.00 97.40 -0.02 2 97.42
4 FY27 - 94.70 94.50 +0.11 2 94.39
Queensland - Caps
Caps Bid Ask Last +/- Vol Settle
0 Q324 7.00 9.00 8.25 - 15 8.25
1 Q424 17.25 18.90 19.00 +0.10 1 18.90
2 Q125 42.00 45.25 45.00 - 1 45.00
3 Q225 - 17.00 - - - 17.00
4 Q325 - - - - - 16.84
South Australia - CapStr
CapStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 22.18
1 CY25 - 23.85 - - - 23.74
2 FY26 - 23.30 - - - 23.19
3 CY26 21.50 23.50 22.99 - 10 22.99
4 FY27 - - - - - 22.78
South Australia - PQtr
PQtr Bid Ask Last +/- Vol Settle
0 Q324 95.0 - - - - 95.0
1 Q424 100.0 - - - - 100.0
2 Q125 160.0 - - - - 160.0
3 Q225 100.0 - - - - 100.0
4 Q325 90.0 - - - - 90.0
South Australia - PkStr
PkStr Bid Ask Last +/- Vol Settle
0 FY25 - - - - - 113.53
1 CY25 - - - - - 107.21
2 FY26 - - - - - 69.79
3 CY26 - - - - - 54.00
4 FY27 - - - - - 54.00
South Australia - BMth
BMth Bid Ask Last +/- Vol Settle
0 Jul24 - - - - - 129.78
1 Aug24 - - - - - 129.78
2 Sep24 - - - - - 129.78
3 Oct24 - - - - - 67.32
South Australia - BQtr
BQtr Bid Ask Last +/- Vol Settle
0 Q324 - 154.75 - - 2 129.78
1 Q424 - - - - 2 67.32
2 Q125 108.50 120.00 - - 2 117.08
3 Q225 - 123.50 - - 2 116.12
4 Q325 - - - - - 119.00
South Australia - BStr
BStr Bid Ask Last +/- Vol Settle
0 FY25 - - 107.50 - 2 107.50
1 CY25 - - - - - 105.90
2 FY26 - 108.00 - - - 104.38
3 CY26 - - - - - 103.45
4 FY27 100.00 105.00 - - - 103.14