javascriptpythonweb-crawlerurlopen
Why missing content/values when using urlopen to crawl data?
I just use following Python codes to crawl data
html = urlopen("https://www.hkex.com.hk/?sc_lang=en").read().decode('utf-8')
print(html)
But I missed the content and only got
<div class="type value"></div>
My goal is to get
<div class="type value">HK$24,225M</div> or HK$24,225M
Solution
The data on this website is updated using JavaScript. Try to press Ctrl+U in your browser.
In this example, the data is fetched from https://www.hkex.com.hk/eng/csm/script/data_NBSZ_Turnover_eng.js or data_SBSZ_Turnover_eng.js. (I don't know what you need)
In the future look at the "Network" tab in the developer tools, you can probably find what you need there.
- Convert the greatest binary number (by number of binary digits) to decimal
- Reading excel file into array using javascript
- Why is Vue's Composable not Isolated?
- Refused to execute script, strict MIME type checking is enabled?
- contentEditable - Firefox <br /> tag
- How to listen for a page $emit in a nuxt layout?
- Why am I getting 'Cannot GET' in localhost for Vue npm run serve
- I can't use dynamic components in Nuxt 3
- Get all files within a folder containing string, push filenames to array and return array using Node FS
- How to fix bubbles when drawing with canvas?
- how to find duplicate number in an array and increment the duplicate value using javascript
- How to enable a checkbox with Javascript?
- Check/Uncheck checkbox with JavaScript
- Symfony 7.0 AssetMapper: JS Files Only Load on Initial Page Load or After F5 Refresh
- Checkbox checked not passing an actual value?
- Err Only plain objects, and a few built-ins, can be passed to Client Components from Server Components. Classes or null prototypes are not supported
- How wait to run JavaScript function until after the DOM is ready / loaded?
- MUI TextField Background Color for auto complete
- Regular expression isn't matching all hyperlinks in a string
- downloading file-stream from browser get request
- Problem with selecting a specific web element with Playwright in Python
- how to keep class attribute stored for Leaflet.js markers
- Index of last vowel in a string
- Delete a specific user from Firebase
- How to access DOM elements by type? (similar to getElementsByName())
- How to use a regular expression in a jQuery selector?
- How to getElementByName() with a regex?
- "user friendly" vimeo embedding, get embed code from url
- Is there a way to achieve currying with a local variable
- "BigInt literals are not available when targeting lower than ESNext"