Analyzing Traffic - Your Online Success is within Log Files - Part 3 Fixing Errors Revealed by Web Traffic Logs

We have covered ways that the web traffic logs can help tailor your marketing campaign and give better insight into the action of visitors. Another way to use traffic logs to benefit your online store is tracking down the cause of errors and correcting them. With any site of significant size, errors will always crop up and can be difficult to track down. By carefully analyzing your server's log files, you can pinpoint these errors and correct them.

404 - File Not Found Errors
These are the most common errors that you will see. Most of these errors are actually caused by web browsers and search engines looking for files that do not exist. Search engine spiders will look for a file called 'robots.txt'. This file instructs the search engine what folders it can crawl, and which ones are off limits. If you do not have a robots.txt file then the crawler will completely crawl your site. This will also lead to a 404 error being generated since the robots.txt file does not exist. The easiest way to correct this error is to create a robots.txt file in the root of your website.

This file should contain the following information:

User-agent: *
Disallow:

This file will tell the search engine crawlers they have full access to your entire site, and will stop their request for robots.txt from generating a 404 error.

Your visitor's web browser is another common cause of 404 errors. Many browsers including Internet Explorer try to retrieve a file called favicon.ico. This file is an icon the browser then uses when adding the site as a bookmark or in the address bar. The fix for this is pretty straightforward as well. Simply create an icon within the graphics program of your choice. Name this file favicon.ico and place it in the root of your site.

Finally add the following line to your HTML pages in the section of the document.

Visit ScholarshipCoach.com to see an example of a site that I designed using a favorite icon.

Once these two most common causes of 404 errors are eliminated, tracking down the causes of other broken links and missing files becomes much easier.

403 - Forbidden
These errors are most commonly caused by visitors attempting to access a password protected directory. If they are denied access for not having the proper username/password then a 403 Error is generated

500 - Internal server error
This error represents a problem with the software on your webserver. These errors are serious since they are often errors that prevent orders from going through. Analyzing the server logs and the times of these errors can help your programmers fix these critical errors.