I wanted to detect whether request is from prerender.io(library used to render the angularjs application when request is from crawlers) or from real users.If request is from prerender, then i have to redirect to the page which is only designed for SEO purpose.

I tried setting cookies to detect but it doesn't work since prerender.io executes javascript code and even the cookie/session storage works in prerender.io.

After some research i found that we can detect user agent and since prerender.io calls site in headless browser(i.e phantomJS)

if (/PhantomJS/.test(window.navigator.userAgent)) {
          //  console.log("PhantomJS environment detected.");
 } else {
        //  console.log("PhantomJS environment not detected.");    
  }

but is it a permanent/proper/best fix for this issue? is there any other solution?

I wanted to detect whether request is from prerender.io(library used to render the angularjs application when request is from crawlers) or from real users.If request is from prerender, then i have to redirect to the page which is only designed for SEO purpose.

I tried setting cookies to detect but it doesn't work since prerender.io executes javascript code and even the cookie/session storage works in prerender.io.

After some research i found that we can detect user agent and since prerender.io calls site in headless browser(i.e phantomJS)

if (/PhantomJS/.test(window.navigator.userAgent)) {
          //  console.log("PhantomJS environment detected.");
 } else {
        //  console.log("PhantomJS environment not detected.");    
  }

but is it a permanent/proper/best fix for this issue? is there any other solution?

• javascript
• angularjs
• seo
• prerender

4 Answers 4

In their FAQ, you can see the UserAgent they use:

What is your crawler's user agent?

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3159.5 Safari/537.36 Prerender (+https://github./prerender/prerender)

You can use userAgent to know if the request es from their crawler.

To detect request from GoogleBot:

You could use IP range ans UserAgent as suggested by Google in the following article in order to redirect in case of a GoogleBot.

https://support.google./webmasters/answer/80553

For the other cases you could use the UserAgent only, similarly to what you are doing now.

You can check for "Prerender" in the user agent but I would not remend changing the content of the page when detecting the Prerender.io browser because that is considered cloaking.

You want to serve the crawlers the raw HTML of the same page that users see.

Prerender detection

According to the Prerender documentation the best is to search for Prerender in the user-agent and not for any other parts of the string because they constantly upgrade their Chrome versions so these can change at any time.

var isPrerender = navigator.userAgent.toLowerCase().indexOf('prerender') !== -1;

https://docs.prerender.io/article/7-faq

permanent/proper/best fix

I am adding for pleteness and because because GoogleBot detection was mentioned in other answers. This is how a project I know is set up:

• When your web app is called the user-agent needs to be checked and if the client is a bot you should make a call to prerender and you need to pass the url of the requested page.

• Prerender will go to the page, render it and returns a HTML which should be sent back in the response.

• Your url will be visited two times once by the bot and then by prerender. You can detect prerender as well and could make things easier by having workarounds for things don't work for headless browsers, remove splash screens and setting the window.prerenderReady flag https://docs.prerender.io/article/11-best-practices.

• If the user-agent is not a bot nor prerender then you do things as usual and serve your angular app without tweaking.

The bot detection and prerender invocation happens on the server side or in a solution which intercepts http calls before they get to your app. There are the ones I am familiar with and the list is by no means exhaustive:

• C# http module: https://github./greengerong/Prerender_asp_mvc/blob/master/Prerender.io/PrerenderModule.cs
• Cloudflare worker: https://github./prerender/prerender-cloudflare-worker/blob/master/index.js