crawl algorithm for sitemap generation tool - DevHunters.com l Webmaster Forum - Web Advertising - Web Design

arathi · 01-24-2008, 05:33 AM

hi

I'm in the process of developing a online sitemap generator tool using php.

I've studied all the necessary prototypes and the requirements for the tool.

But i couldn't come to a conclusion on the algorithm the tool should be using for crawling..

What would be the be the best algorithm (for searching and adding links) for the crawler program?

Please help me with this..

SpOrTsDuDe.Reese · 01-24-2008, 08:14 AM

I would recommend either C#, Ruby On Rails, Or, my personal favorite, Python.

J0ker · 01-24-2008, 03:49 PM

He said he is using php. You could make it make a sitemap of all the page, but that wouldnt work well with sites with massive page amounts.

Have A Look At Some Of Our Webmaster Related Sponsors!
Script Directory	Advertise Here	Market Leverage	Free Templates	3D Guide-characters	Advertise Here

01-24-2008, 05:33 AM	#1 (permalink)
arathi New Hunter Join Date: Jul 2007 Posts: 6 iTrader: 0 / 0% Hunter Bux: 0	crawl algorithm for sitemap generation tool hi I'm in the process of developing a online sitemap generator tool using php. I've studied all the necessary prototypes and the requirements for the tool. But i couldn't come to a conclusion on the algorithm the tool should be using for crawling.. What would be the be the best algorithm (for searching and adding links) for the crawler program? Please help me with this.. __________________ Script Directory World Internet Statistic Indian colleges

01-24-2008, 03:49 PM	#3 (permalink)
J0ker Administrator Join Date: Apr 2007 Posts: 165 iTrader: 0 / 0% Hunter Bux: 1,520.00	He said he is using php. You could make it make a sitemap of all the page, but that wouldnt work well with sites with massive page amounts. __________________ Bidding Directory \| Web Directory \| Blurpalicious \| Gaming Forum \| Stumble Upon

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

01-24-2008, 08:14 AM	#2 (permalink)
SpOrTsDuDe.Reese Senior Moderator Space Invaders Champion! Breakout Champion! Moon Lander Champion! Copter Champion! Archery 2000 Champion! Curveball Champion! Zombie Survival Champion! Pacxon Champion! SD-Pong Champion! Tennis Challenge Champion! Rabbit Hunting Champion! Let It Ride Champion! Hot Shots Champion! Van Wilder Beer Pong Champion! Tournaments Won: 2 Join Date: Jan 2008 Location: Whiteford, MD Posts: 1,125 iTrader: 1 / 100% Hunter Bux: 1,786.00	I would recommend either C#, Ruby On Rails, Or, my personal favorite, Python.