Search code examples
asp.netseosearch-enginegoogle-crawlers

How to make search engines crawl tens of thousands of pages in my website


I have a large amount of items that each item has a page like this site/item_show.aspx?id=The_ID_here there are tens of thousands of items and each day nearly two thousands are added. Furthermore each item has a description in its page so for each item, its page should be crawled by search engines.

My question is with this amount of data described: How can I generate sitemaps or anything like that to make all items visible by google and other search engines?

It is clear that I cannot show all items in the first pages but I can make pages that simply just contains the link of items and provide tens of them each page for just search engine. Would it work or is it anything better to do for making items be indexed by google?


Solution

  • essentially there are 3 methods which will help you with the Mass-Indexing : 1. Create an XML SiteMap for all of your pages and link to it from you HomePage. 2. you should have Google Webmaster Tools Installed and you can load that same XML file into it. 3. Have an Organized Categories structure - depending on the type of your site think about a logical Categories Structure for example in eCommerce stores all the products are categorized by product Main-Category then a product Sub-category and sometimes by Brands etc... - of course you should do this via your Shopping-cart platform - Just remember that if you begin chaging the URLs-structure you'll need to take care of all the Redirects from the old URLS to the new ones.