I am going to post what some may consider a generic/default robots.txt for standard Zen Carts installed in to the root of your domain. I strongly suggest you read and learn to properly use the robots protocol and directives instead. I will take no responsibility for this, as it is a suggestion and you have been warned that every Zen Cart is different. Additionally, this WILL NOT work at all if you have rewritten your page’s urls.
This is really a long and tiresome debate… I personally have argued this point for a long time to no avail with Zen Cart forum posters… Even linked to Sebastian for verification. But, alas, the debate drags on.
Blocking my page in robots.txt will prevent indexing.. This is just not true, your robots.txt is a tool used to prevent crawling… not indexing. While blocking a page in robots.txt will prevent well behaved spiders from crawling these pages, it will not prevent indexing, remove a page from the index, nor block the flow of PageRank to the page.
Google announced yesterday that they have launched a fresh new design for webmaster tools. If you have never used Google’s webmaster tools the you can get started with the new interface right off. If you are already using these excellent tools from Google, then login to find a link to “Checkout the New Look” in…