Mobile-First Indexing


So logging in to webmaster tools following an email from Google about Mobile-First Indexing on one of our sites, we found the following documentation.

Mobile-first indexing enabled for https://mysite.com/
May 17, 2018

To owner of https://mysite.com/,

This means that you may see more traffic in your logs from Googlebot Smartphone. You may also see that snippets in Google Search results are now generated from the mobile version of your content.

Background: Mobile-first indexing means that Googlebot will now use the mobile version of your site for indexing and ranking, to better help our (primarily mobile) users find what they’re looking for. Google’s crawling, indexing, and ranking systems have historically used the desktop version of your site’s content, which can cause issues for mobile searchers when the desktop version differs from the mobile version. Our analysis indicates that the mobile and desktop versions of your site are comparable.
For more information:
• Monitor your index coverage using the Index Status report.
• Monitor your impressions, clicks, and site appearance using the Performance report.
• If you have any additional questions, drop by our Webmaster forum for more help – mention message type [WNC-20058194] or our public events.

It is VERY important to note that this took more than a year to happen since the site was updated to a responsive template.

Some of the language in the documentation I find interesting: “You may also see that snippets in Google Search results are now generated from the mobile version of your content” and “Our analysis indicates that the mobile and desktop versions of your site are comparable”.

Google Search results are now generated from the mobile version of your content: This is of no concern to me as the content displayed between the versions is near exact, but what if it wasn’t? What if the mobile version was lean on content…. That could be a big problem. What if your mobile version is a separate version all together? Worse, what if people start landing on your mobile site by default and are not properly redirected (this is a problem I have noticed with browser intricacies)? I think if I had a mobile cart instead of a responsive site this would move me to get responsive! The documentation is here.

Google says: “If you have a site configuration where the primary content and markup is different across mobile and desktop, you should consider making some changes to your site.” (BINGO)

Google Also says: “If you are a site owner who has only verified their desktop site in Search Console, please add and verify your mobile version.” (I would be looking to go responsive instead)

And: “Your mobile site should contain the same content as your desktop site. If your mobile site has less content than your desktop site, you should consider updating your mobile site so that its primary content is equivalent with your desktop site. This includes text, images (with alt-attributes), and videos – in the usual crawlable and indexable formats.”

Our analysis indicates that the mobile and desktop versions of your site are comparable: Geee, great. So a robot using a mathematical algorithm decided this? I find that concerning. Also, what happens if Google didn’t find the version “comparable”?

In reality, we already know Google and other search engines are favoring mobile ready sites with more favorable rankings, but in beginning in July 2018 Google will begin (take this with a grain of salt as I’m pretty sure it started a year ago) not favoring site’s without mobile capability.

Google Says: “Google tells the webmasters of sites that are not yet mobile-optimized to not panic yet. “If you only have desktop content, you will continue to be represented in our index,” assures the Google announcement.”

I Say: uhuh, sure……

So aside from being mobile friendly and apparently fast, what else should be done?

Google Says:

  1. Verify both versions of your site in Search Console to make sure you have access to data and messages for both versions. Your site may experience a data shift when Google switches to mobile-first indexing for your site.
  2. Check hreflang links on separate URLs. When you use rel=hreflang link elements for internationalization, link between mobile and desktop URLs separately. Your mobile URLs’ hreflang should point to mobile URLs, and similarly desktop URL hreflang should point to desktop URLs. Example of hreflang for separate URLs:
    link rel=”alternate” hreflang=”es” href=”https://m.example.com/es/”
    link rel=”alternate” hreflang=”fr” href=”https://m.example.com/fr/”
    link rel=”alternate” hreflang=”de” href=”https://m.example.com/de/”
    link rel=”alternate” hreflang=”th” href=”https://m.example.com/th/”
  3. Ensure your servers have enough capacity to a handle potential increase in crawl rate on the mobile version of your site.
  4. Verify that your robots.txt directives work as you intended for both versions of your site. The robots.txt file lets site owners specify which parts of a website may be crawled or not. In most cases, sites should use the same directives for both mobile and desktop versions of their sites.
  5. Make sure you have the correct rel=canonical and rel=alternate link elements between your mobile and desktop versions.

Let’s break these items down in to layman’s and Zen Cart terms, shall we?

  1. Verifying both version of your site is pretty simple, works the same as verifying for both HTTP & HTTPS. This is NOT necessary unless your mobile cart is on a different url like mysite.com/mobile/ for example.
  2. hreflang is ALWAYS necessary and important for usability. If your site offers other languages make sure to check in the source that the other than default languages are correctly represented in the hreflang tag. If you are using a url rewriter in your Zen Cart this is going to be a challenge, but not impossible.
  3. If your server is performing well for your current usage, then I wouldn’t concern myself too much with this. However, if your running hot already, then it’s time to upgrade.
  4. Your robots.txt should NOT be blocking anything really. robots.txt only blocks crawling, not indexing, so if there are pages you want to prevent search engines from indexing, then you have to block them with a nofollow tag. Zen Cart makes this easy with the robots skip function. (Edit /includes/languages/YOUR_LANGUAGE/YOUR_TEMPLATE/meta_tags.php, adding those not-to-be-indexed pages to the ROBOTS_PAGES_TO_SKIP definition.)
  5. Your rel=”canonical” should be good to go, but check it anyhow in your page’s source.

Now I have some items to add:

  • Content is still king! Make sure your mobile version has some meat on the bones. Make sure you are adding new content regularly.
  • Test your mobile pages with the Mobile-Friendly Test to see if our systems detect your website as compatible for mobile users. Test a variety of pages every couple of months.
  • Structured data is going to help everyone understand your website better. This addon for Zen Cart structured data is easy to install and works great and it includes Facebook, Twitter and alias profile page entries as well.
  • Make sure your search engine sitemap is correct and up to date. I highly recommend you use a program that actually crawls your website as it can be a real eye-opener for issues that exist. We can help with that here.
  • Make sure you are properly using switchboard tags if your mobile version is separate. Here is Google’s documentation, but I still say responsive is the way to go.
  • SPEED! Google, the other search engines, humans and the world have a need for speed. Not every site is going to be crazy fast, but make sure you are doing the best you can. Images should be no more than 750px in any direction and crunched. Limit the number of stylesheets and javascripts by optimizing them and combining files. Move scripts not required for every page to only the page that requires them and move any you can to the bottom of the page.
  • Use HTTPS full time. If you haven’t switched your Zen Cart for all pages HTTPS you need to. Remember, browsers are sending visitors to the HTTPS versions anyhow! Here’s how
  • Get friendly with Google Webmasters Tools…. as there is much to learn Danielson
  • Lastly, I would think twice about using a Foundation template over a Bootstrap template. It’s concerning that Foundation has to “know” the device to render correctly, whilst Bootstrap just says “hey man, how big is your screen”… Cool bro, got you.