how to crawler to save mp3 urls in asp.net mvc web application

Question

i want to create an application that can crawl different websites and collect mp3 urls on the basis of given query.

user can search song title if results are available in database then show related song url otherwise crawl websites and find related result save in database and shows resutls to user.

Answer 1

I would give HtmlAgilityPack a go (you can install using package manager).

A simple example of how to start:

string url = "http://www.google.com";
HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(url);

When you have loaded a document, you can inspect it:

foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//a[@href]"))
{
     if (node.Attributes.Contains("href"))
     {
         Console.WriteLine(node.Attributes["href"].Value);
     }
}

The above should print all the URLs it can find for the anchors.