JavaScript SEO

How Google Processes JavaScript Content

Is Javascript bad for SEO? Honestly, it depends on several factors. But when JS is primarily being used to render important pages, the answer’s yes! Let’s dig deeper into how Google processes JavaScript and its impact on your site’s SEO.

Google bots do not crawl JavaScript pages in the traditional sense. They can only crawl the static HTML source code. So, when a bot comes across content rendered with JS, it has to follow a few steps to render the content – crawling, rendering, and indexing.

Here’s what happens when bots reach a regular HTML/ non-JS page.

• The bot downloads the raw HTML file for your page.
• It passes the HTML to Caffeine (the indexer) to extract all the links and metadata.
• The bot continues to crawl all the discovered links.
• Caffeine indexes the extracted content that is further used for ranking.

Fortunately, Caffeine can now render JS files like a browser would (through Google’s Web Rendering Service). So, here’s how Google with its WRS reaches your JS-powered pages.

• The bot downloads the raw HTML file for your page.
• The first indexing occurs instantly without the rendered content. Simultaneously, Caffeine works on rendering the JavaScript page.
• The extracted links, metadata, and content are passed back to the bots for further crawling.
• The extracted content is indexed during the second indexation process and used for ranking.

The following image shared by Google in its brief guide on JavaScript SEO summarizes the process. However, it’s important to note that the bots will crawl and index any HTML page it finds along the way.

Source

Let’s look at each of these in detail.

Crawling

To begin with, the Google bot fetches a URL for a page from the crawling queue and checks if it allows crawling. Assuming that the page isn’t blocked in the robots.txt file, the bot will follow the URL and parse a response for other URLs in the href attribute of HTML links.

If the URL is marked disallowed, then the bot skips making an HTTP request to this URL and ignores it altogether.

Rendering or Processing 

At this stage, the URL is processed for JavaScript. The page is placed in the render queue.

Once it’s rendered, the bots will add the new URLs it discovers to the crawl queue and move the new content (added through JS) for indexing.

There are two types of rendering – server-side and client-side rendering.

• Server-Side Rendering (SSR)

In this type of rendering, the pages are populated on the server. So, every time the site is accessed, the page is rendered on the server and sent to the browser.

Simply put, when a visitor or a bot accesses the site, they receive the content as HTML markup. So, Google doesn’t have to render the JS separately to access the content, thus improving the SEO.

• Client-Side Rendering (CSR)

Client-side rendering is a fairly recent type of rendering that allows SEOs and developers to build sites entirely rendered in the browser with JS. So, CSR allows each route to be created dynamically in the browser.

CSR is initially slow as it makes multiple rounds to the server but once the requests are complete the process through the JS framework is quick.

Indexing 

At this stage, the content (from HTML or new content from JS) is added to Google’s index. So, when a user keys in a relevant query on the search engine, the page will appear.

Top JavaScript SEO Strategies

Knowing how to tackle the challenges and get the most of  JavaScript is vital for modern SEOs. Here are a few strategies that can help you navigate the world of JavaScript SEO.

1. Continue Your On-Page SEO EffortsAll the on-page SEO guidelines related to the content, title tags, meta descriptions, alt attributes, and meta robot tags among others still apply.For instance, unique and descriptive titles and meta descriptions help users and crawlers process the content with ease.Similarly, using short descriptive URLs is one of the best on-page SEO tactics that get the most clicks as they match the search query.
2. Fix Duplicate Content Multiple URLs with the same content is a common occurrence with JavaScript heavy sites. Such duplicate content issues are caused by capitalization, IDs, and parameters with IDs among other factors.Duplicate content doesn’t add any value to a site’s SEO; in fact, it confuses the crawlers and increases the risk of important pages being ignored by Google. Make sure you choose a version you want the crawlers to index and set canonical tags to fix duplicate content.
3. Use Meaningful HTTP Status CodesHTTPS status codes like a 404 or 401 are used by the bots to determine if something went wrong during the crawling process. Use meaningful status codes like the ones shared below to inform the bots whether a page needs to be crawled or indexed.

Source

For instance, you can use a 301 or 301 status code to tell Google bots that the page has moved to a new URL. This will help the bots update the index accordingly.

1. Employ Clean URLs and MarkupJavaScript can dynamically alter or update content or links on a webpage, causing confusion for the search engine bots. Therefore, it’s important to use clean URLs and links with the <a href> attribute. This will ensure that your pages are easily discoverable.The same applies to markup. So, make sure that the key markup is structured clearly and is included in the HTML for page titles, meta descriptions, canonical tags, alt attributes, and image source attributes among others.
2. Improve the Page Loading Speed Page speed is one of the critical ranking signals used by Google to rank pages. A website that loads quickly helps bots with the overall indexation process and improves UX. However, this can be an issue with JavaScript. Mitigate this issue by using lazy loading for certain components.Images often eat up the bandwidth and negatively impact the site performance. Use lazy loading to load images, iframes, and non-critical content only when the user is about to see them.However, if not implemented correctly, lazy loading can hide critical content from the bots. Use this lazy-loading guide by Google to ensure that the bots crawl all your content whenever it’s visible in the viewport.
3. Conduct a JS Website AuditThis means conducting a routine manual inspection of the individual elements using the Google Developer Tools from Google Chrome and the Web Developer Extension for Chrome. Here’s what you can include as a part of this audit.

• Visual Inspection: This will help you get an idea of how a user views your site. Check elements like the visible content on the site, hidden content, content from third parties, and product recommendations. The goal is to make these elements crawlable.
• Check HTML Code: Turn off the CSS, JavaScript, and cookies to check the balance code that’s controlled by JS.
• Check the Rendered HTML: Load the site with deactivated JavaScript and CSS. Next, right-click and choose ‘Examine element’ in the Chrome menu. Click on the HTML tag on the right-side for the ‘Options’ menu.

Choose ‘Copy External HTML.’ Finally, insert the code into the editor to allow the code to be indexed by the search bots. You can also test JavaScript with the Google Search Console using the URL inspection tool.

Finally, make sure you avoid the common errors in JavaScript SEO we discussed earlier in this post.

JavaScript SEO – Frequently Asked Questions

Here are a few questions we commonly come across on JavaScript SEO. We have attempted to share satisfying answers to each. But in case you have any queries on JS SEO, feel free to get in touch with our team!

• What Happens If I Don’t Do JS SEO? 

Though Google is improving its ability to index JS content, there’s still no surety that the content will be indexed. There are several caveats in Google’s ability (not even talking about other search engines and social channels here!) to process JavaScript content.

This makes it imperative for webmasters depending on JavaScript to make sure that search engines can access, render, and index their content with ease.

• Can I Do Something to Reduce JavaScript’s Impact on Web Performance? 

Yes! Techniques like code minifying and compressing, caching, and tree shaking can be used to reduce the necessary bandwidth usage and boost web performance. For instance, tree shaking is a form of dead code elimination that can significantly reduce JS payloads and improve the site’s performance.

• In Light of JS SEO, Is PWA SEO Required? 

Using JS as a PWA or  Progressive Web Application framework ensures the best possible UX. However, there have been concerns about creating a crawler-friendly JS application.

PWA is a form of application software that is delivered through the web and built on web technologies like HTML and JS. Hence, ideally, all the recommendations applied to JS sites should work with PWAs too. However,  the Google Web Rendering service has Service Workers (one of the pillars of the PWA architecture) disabled.

Like JS SEO, PWAs also need to be optimized as they need to be SEO compatible and indexed properly. PWA SEO is possible; however, Google should be able to render JS pages for it to be able to view the content present in the PWA.

Talk to an expert SEO agency that can help get your PWA indexed and make them discoverable.

• How Mobile-First Indexing Affects JS Pages?

Mobile-first indexing is Google’s indexing system that first considers the mobile version of a website. So, if the desktop version of your website is fully optimized but your mobile version uses poor JavaScript SEO practices, your website ranking will suffer.

So, it’s critical to audit your site’s desktop and mobile versions, allowing Google bots to easily index your pages.

• If Google Cannot Deal with the Client-Side Rendered Website, Can I Serve It a Pre-Rendered Version of My Website? 

Yes! Serving Google bots a pre-rendered version is a workaround solution to allow the crawlers to successfully and immediately process JavaScript content. The process is referred to as dynamic rendering and it’s officially accepted and supported by search engines like Google and Bing.

Check out this video for more information on dynamic rendering.

• Google Constantly Claims to Fix the Issue Related to the Crawlers Processing JS Content. So, Will JS SEO Not Be Relevant in the Future? 

As new technologies and frameworks are introduced, JS SEO will only get more technical, increasing the demand for SEO experts who can optimize JS content. Already,97 percent of all websites use JavaScript. Mastering JS SEO will only help you supersede competition and improve your site’s ranking.

Google’s URL Inspection Tool helps SEOs determine whether or not Google is rendering its web pages. You can run live tests to see real-time JS warnings and errors that were coming in the way of your pages being indexed or discovered.

Mobile-Friendly Test

Source

You can use this test to determine whether your pages render on smartphones. For this, you do not need a Google Search Console account. The test also points out the errors or blocked resources stopping crawlers from accessing your content.

• Data Comparison Tools

Tools like Diffchecker, Guesty, and Microsoft Flow can perform a quick analysis of a web page’s source code vis-a-vis its rendered code. This comparison offers insights into how the content changes once it’s rendered.

Further, Chrome Extensions like View Rendered Source helps webmasters compare the raw HTML source of a page to the browser rendered DOM. It also shows the difference between the two.

• SEO Crawlers and Log Analyzers

SEO spider tools like DeepCrawl, Screaming Frog, and JetOctopus can be used to get granular insights on each page and test/monitor rendering at scale.

• PageSpeed Insights 

Google’s PageSpeed Insights tool uses a combination of data from Google Lighthouse and lab data to deliver insights related to speed and page performance. Besides, it offers JavaScript reports that show how a page could be optimized by reducing the JS execution time and minifying JS.

• Chrome DevTools 

If you use Chrome as a primary browser, the debugging and JavaScript testing capabilities of Chrome DevTools can be used to generate a variety of reports. Performance monitoring and identifying JS errors can be easily achieved with this tool.