Search code examples
web-crawlerstormcrawler

How many Crawlers can run simultaneously using storm crawler


I am working on StormCrawler 1.10,archetype 1.11 and Elastic Search 6.4.x. How many crawlers can run simultaneously.


Solution

  • Crawlers == topologies? As many as you want as long as they have different names. Why do you need multiple topologies? You can have a single one dealing with multiple sites