htmlcleaner parse with tags

Question

I try to extract some part of page. I use parser HtmlCleaner, and it remove all tags. Are there some settings to save all html tags? Or maybe is better way to extract this part of code, using something else?

My code:

static final String XPATH_STATS = "//div[@class='text']/p/";

// config cleaner properties
HtmlCleaner htmlCleaner = new HtmlCleaner();
CleanerProperties props = htmlCleaner.getProperties();
props.setAllowHtmlInsideAttributes(false);
props.setAllowMultiWordAttributes(true);
props.setRecognizeUnicodeChars(true);
props.setOmitComments(true);
props.setTransSpecialEntitiesToNCR(true);


// create URL object
URL url = new URL(BLOG_URL);
// get HTML page root node
TagNode root = htmlCleaner.clean(url);


Object[] statsNode = root.evaluateXPath(XPATH_STATS);
for (Object tag : statsNode) {
    stats =  stats + tag.toString().trim();
}

return stats;

thanks for nikhil.thakkar! I do this by JSON. The code may help someone:

    URL url2 = new URL(BLOG_URL);
    Document doc2 = Jsoup.parse(url2, 3000);
    Element masthead = doc2.select("div.main_text").first();
    String linkOuterH = masthead.outerHtml(); 

Answer 1

You can use jSoup parser. More info here: http://jsoup.org/