Crawl-on-demand
This function is only enabled for paying users. You can use this function to refresh
your data in Sitexploration. Though, Siexploration also crawls randomly by itself.
When using the crawl-on-demand function, you control a given crawler’s work pattern.
This is extremely useful when you what to know what others are currently competing
over and what they are competing over in the future. When you use crawl-on-demand,
you order ZookaBot to crawl the url and revisist or discover a sites link graph.
The crawl-on-demand 'behind the scenes' work with three vic crawl containers that
are running concurrently. You, the users’, have your own dedicated container for
the crawl-on-demands. The other two is dedicated to the system, where it periodically
searches the internet for new links. Searches not finished on vic 2 after being
there for 24 hours will be moved to vic 1. If the page is not indexed by Google,
then you should not expect links like these to send any linkjuice. Therefore Sitexploration
removes them from its index and move them to an index cesspool.