Search code examples
pythonbeautifulsoupextractscreen-scraping

How to extract specific data from stcipt var?


The following is part of the html code from a page from which I want to extract the data

<script type="text/javascript">
var productOptInfo = {
        prdNo: "3385792003",
        //lastPrc : 497500,
        optCnt: 5,
        selOptCnt: 3,
        entOptCnt: 2,
        selOptTyp: "01",
        optItemNms: 
        isOptCalc: false,
        isNotOptPrd: false,
        totStockQty: 9999,
        totPrdStckNo: "12951427057",
        totPrdPrc: "0",
        defaultOptQty: "1",
        optCheckStatus: "OK",
        orderOptArr: [],
        orderAddPrdArr: [],
        orderRecmPrdArr: [],
        isHighPrice: false,
        SPLIT_SEPARATOR: ":∥:"
    };

My code is..

import requests
from bs4 import BeautifulSoup
import re

url1 = 'http://www.11st.co.kr/products/3167879989'

req = requests.get(url1).text
soup = BeautifulSoup(req, 'lxml')
js = soup.find_all('script')[27].string
m = re.search(r'var productOptInfo = (.*?);', js, re.S).group(0)
print(m)

How can I extract the value of "optCnt",selOptCnt" and "entOptCnt" ?? I want to get the value 5,3,2


Solution

  • You can install the demjson module: https://pypi.org/project/demjson/ and use it to parse the javascript object - you'd get back a dictionary in this instance.

    # note it's group(1)
    
    m = re.search(r'var productOptInfo = (.*?);', js, re.S).group(1)
    j = demjson.decode(m)
    
    >>> j['optCnt']
    2
    >>> j['prdNo']
    '3167879989'