crawl algorithm for sitemap generation tool
hi
I'm in the process of developing a online sitemap generator tool using php.
I've studied all the necessary prototypes and the requirements for the tool.
But i couldn't come to a conclusion on the algorithm the tool should be using for crawling..
What would be the be the best algorithm (for searching and adding links) for the crawler program?
Please help me with this..
|