How to generate Sitemaps for a large Rails Project

Question

We struggle to find the best solution to generate our sitemap

Right now we have the following models which needs to be displayed in our sitemap(s)

Modification.count => 78719
User.count => 288168
Project.count => 3516
ProjectLog.count => 6165    
Category.count => 272

While we can only have 50k links per sitemap, we need multiple sitemaps. The app is running on multiple servers, so we need to have the sitemaps also on multiple servers.

So here are my ideas:

generate sitemap.xml which contains links to modifications_1.xml, modifications_2.xml etc.

this won't be a problem at all, but unfortunately those entries will change sometimes, so we need to update the last_modification_time

my approach would be:

1. initial generate the files.
2. if some entry is updated, open the XML file, and change the key (nokogiri propably)
3. if new entries are added, add them to the latest xml-file
4. if entry is deleted, delete it from the xml file.
5. every 2 weeks, regenerate all files again.

are there any better approaches to do this?

i checked some available gems but it looks like they don't fit our project size.

Answer 1

seems like this gem can handle it out of the box. sexy.

https://github.com/kjvarga/sitemap_generator