Search code examples
pythonpython-3.xpandasdataframexml.etree

Extract panda dataframe from .xml file


I have a .xml file with following contents:

<detailedreport xmlns:xsi="http://"false">
<severity level="5">
<category categoryid="3" categoryname="Buffer Overflow" pcirelated="false">
<cwe cweid="121" cwename="Stack-based Buffer Overflow" pcirelated="false" sans="120" certc="1160">
<description>
<text text="code."/>
</description>
<staticflaws>
<flaw severity="5" categoryname="Stack-based Buffer Overflow" count="1" issueid="6225" module="Jep" type="strcpy" description="This call to strcpy() contains a buffer overflow. The source string has an allocated size of 80 bytes " note="" cweid="121" remediationeffort="2" exploitLevel="0" categoryid="3" pcirelated="false">
<exploitability_adjustments>
<exploitability_adjustment score_adjustment="0">
</exploitability_adjustment>
</exploitability_adjustments>
</flaw>
</staticflaws>
</cwe>
</category>
</severity>
</detailedreport>

Below is the python program to extract some of the fields from the .xml file under the "flaw" tag. But when I print the fields in python program, they are empty.

from lxml import etree
root = etree.parse(r'fps_change.xml')
xroot = root.getroot()

df_cols = ["categoryname", "issueid", "module"]
rows = []

for node in xroot:
    #s_name = node.attrib.get("name")
    s_categoryname = node.find("categoryname")
    s_issueid = node.find("issueid")
    s_module = node.find("module")

rows.append({"categoryname": s_categoryname,
             "issueid": s_issueid, "module": s_module})

out_df = pd.DataFrame(rows, columns=df_cols)
print(out_df)             #this prints empty.

Expected Output:

 Stack-based Buffer Overflow 6225 Jep

What changes should I do in my program to get my expected output.


Solution

  • from bs4 import BeautifulSoup
    html_obj = BeautifulSoup(string)
    flaw = html_obj.find('flaw')
    [flaw[key] for key in df_cols]
    
    ['Stack-based Buffer Overflow', '6225', 'Jep']
    

    string = '''
    <detailedreport xmlns:xsi="http://"false">
    <severity level="5">
    <category categoryid="3" categoryname="Buffer Overflow" pcirelated="false">
    <cwe cweid="121" cwename="Stack-based Buffer Overflow" pcirelated="false" sans="120" certc="1160">
    <description>
    <text text="code."/>
    </description>
    <staticflaws>
    <flaw severity="5" categoryname="Stack-based Buffer Overflow" count="1" issueid="6225" module="Jep" type="strcpy" description="This call to strcpy() contains a buffer overflow. The source string has an allocated size of 80 bytes " note="" cweid="121" remediationeffort="2" exploitLevel="0" categoryid="3" pcirelated="false">
    <exploitability_adjustments>
    <exploitability_adjustment score_adjustment="0">
    </exploitability_adjustment>
    </exploitability_adjustments>
    </flaw>
    </staticflaws>
    </cwe>
    </category>
    </severity>
    </detailedreport>'''