Feeding the Spider in Order to Earn Revenue

The internet is big, really big. And most all of it has been cataloged at some point in some way, by an automatic program using algorithms that make any formulas you learned in school look like child’s play.

Starting at the very beginning, some definitions are in order.

What are spiders?

Spiders: Spiders are simply programs, they don’t have legs and such and despite their persistent and unrelenting drive in indexing backlinks and everything else they can find, they aren’t nearly as unsettling as the physical variety. A spider has many other names as well, but essentially it is a small software / robot / program containing an algorithm that builds lists of all the words found on the websites it surveys. The spider notes what words were contained within a given page, and beyond that it notes where the words were found.

What is Web Crawling?

Web Crawling: Quite simply, web crawling is the act of the spider being deployed and canvassing the entirety of the internet, indexing and listing everything it finds in sequence, then returning the data thus cataloged to its home servers. This process is at the same time completely chaotic and predictable to some extent. Spiders will follow traffic, starting on the most heavily used servers as they have been deployed and listing everything found there. After all the pages of that server are completed, it moves to repeat the process using each of the destinations on the backlink list found on the pages and sites it just finished cataloging.

Winning the “Popularity Contest”

In essence, the entire internet is built on a popularity contest model, with page ranking being paramount in the race for success. Getting noticed is usually one of the major factors of a website’s reasons for existence. Every site, from a local charity trying to save a historical building to an international business giant is looking to leverage their website for exposure and probably hoping to drive revenue. The main way to do this is to generate high quality content, which in turn will allow high quality link building and increase traffic to the site and ultimately enhancing exposure.

Now one of the early questions many startups often ask is ‘should I buy backlinks’ and, unfortunately there is no hard and set answer to this. Services are available that will provide backlink solutions for startup websites, and can even provide advanced SEO services for a commensurate fee. But the simple fact remains: if the content is there, the traffic will come.

In marketing, the idea that presentation is half the battle is often skewed such that presentation is treated as the holy grail of generating results, while from a business perspective the truth is that a quick sale to an unsatisfied customer based on shiny marketing tactics is not a good sale. Repeat customers that are going to bring others back with them is the gold mine you are looking for, and to do that, all the fancy marketing in the world won’t be able to do a thing unless the underlying content is there.

Consider a real world example if you will. A comic book printed on high gloss paper, with lots of eye candy displayed proudly on the front cover entices you to buy the comic, and you open it to find an engaging story line relevant to the cover material. At the end of the story you’re satisfied and want more of the same; you will probably recommend the comic to like minded individuals and will not be shy about picking up the next comic from the same set when it is released. This is a good result, one which marketers and companies in general are looking for.

Now, take the same situation, where the comic’s marketing has convinced you to pick it up, only to find that the story was hard to follow, the internal graphics were of poor quality and your overall impression was such that you never finished the book. The company did manage to get your initial purchase costs, but they have lost out on all future sales due to a lack of follow through with the content. Evidently this is a complete fail as far as good business is concerned.

Final Thoughts

When it comes to generating traffic for a site, an initial marketing blitz via buying links and paying for traffic might be a good investment, but the truth is that getting properly assessed by the search engine spiders, with well designed pages that provide consistent traffic to quality content is the only really viable long term solution. The content needs to be there when the customers who clicked on the inbound links to your site arrive and look around. If there is nothing for them or if the product is of poor quality, then of course they will find other places to be. It takes fractions of a second to lose a customer’s interest on the internet. Engaging visitors with quality content and product as well as an enjoyable experience is what gains a site popularity and traffic. Traffic and popularity are what drives page ranking, and page ranking is often what drives revenues, be it through product sales or simply advertising revenue.