Search code examples
pythonparsingpython-2.7html-parsingbeautifulsoup

Using Beautiful Soup to get data from non-class section


I am still very novice and learning python and beautiful soup. I have gotten hung up on how to get text from a non-class piece of HTML.

This is the snippet of HTML I'm working with:

<section class="userbody">
    <script type="text/javascript"></script>
    <figure class="iw">
        <div id="ci">
            <img id="iwi" title="image 2" alt="" src="http://images.craigslist.org/00C0C_daJm4U9yU5B_600x450.jpg" style="min-width: inherit; min-height: 450px;"></img>
        </div>
        <div id="thumbs"></div>
    </figure>
    <div class="mapAndAttrs">
        <div class="mapbox">
            <div id="map" class="leaflet-container leaflet-fade-anim" data-longitude="-84.072447" data-latitude="33.908534" tabindex="0">
                <div class="leaflet-map-pane" style="transform: translate(0px, 0px);"></div>
                <div class="leaflet-control-container">
                    <div class="leaflet-top leaflet-left"></div>
                    <div class="leaflet-top leaflet-right"></div>
                    <div class="leaflet-bottom leaflet-left"></div>
                    <div class="leaflet-bottom leaflet-right">
                        <div class="leaflet-control-attribution leaflet-control"></div>
                    </div>
                </div>
            </div>
            <div class="mapaddress">

                Some Address

            </div>
        </div>
        <div class="attributes"></div>
    </div>
    <section id="postingbody">
            some posting info
            <br></br>
             more posting info
             <br></br>
    </section>
    <section class="cltags"></section>
    <div class="postinginfos"></div>
</section>

I have been able to pull the address information:

     for address in soup.findAll("div", { "class" : "mapaddress" }):
       addressText = ''.join(address.findAll(text=True))

It appears findAll() doesn't work for tags that have don't have a class as I tried doing in

     for post in soup.findall("section", { "id" : "postingbody" }):
       postText = ''.join(post.findAll(text=True))

How would grab the text in section id="postingbody"?


Solution

  • Well you can do the following, taking into consideration that s is the html string:

    from bs4 import BeautifulSoup
    
    soup = BeautifulSoup(s)
    print soup.find(attrs={'id' : 'postingbody'})
    

    Output:

    <section id="postingbody">
                some posting info
                <br/>
                 more posting info
                 <br/>
    </section>