Monday, September 08, 2008

HTTP Referer Crawl

Application Download

Download: HTTP Referer Crawl
Microsoft .NET Framework 3.5 is required.


Basically, this application will continuously crawl the web using your defined website as the HTTP Referer in the HTTP Headers. Simply put in a Google search term related to your site, and hit start. The crawler will start by finding Google results for your search term, and as it visits those sites, it will constantly grab more sites to visit, without any interaction from you whatsoever!

How does this generate traffic? There are two basic ways:
1. When someone looks at their log files and sees your site as a referer, they may check your site out to see where you linked to them.
2. Some sites publish referer information publicly, which eventually will get picked up by search engines and other bots.

Is this guaranteed to boost your traffic? Not necessarily. It really all depends on the sites this crawler ends up hitting. But hey, it's free, and easy to use, so why not give it a shot?

Screenshot: