Search code examples
javawebnoise

Noise removal from webpages


I am trying to do noise removal from web page . but as I am new to this so i don't know from where to start. so if anybody can give me any information on how to do it , it will be very helpful.

Noise - links , copyright notices etc.

Sudhanshu


Solution

  • Boilerpipe seems like what you're asking for: http://code.google.com/p/boilerpipe/

    The boilerpipe library provides algorithms to detect and remove the surplus "clutter" (boilerplate, templates) around the main textual content of a web page.

    The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual problem settings.