I have this issue with node-fetch which returns error:

ERR FetchError {
      message: 'maximum redirect reached at: .svg',
      type: 'max-redirect'
    }

I am aware that there is already similar topic at What does the 'maximum redirect' error in fetch-api means? but this doesn't solve my issue, these parameters change absolutely nothing in behavior.

This error occurs for any request to this website, others seem to work fine. I also tried to set different user-agent header, but it doesn't help either. Website seem to work normaly when just opening from the browser.

I have this issue with node-fetch which returns error:

ERR FetchError {
      message: 'maximum redirect reached at: https://www.alsa.es/o/Alsa-main-theme/images/web2020/logo-alsa.svg',
      type: 'max-redirect'
    }

I am aware that there is already similar topic at What does the 'maximum redirect' error in fetch-api means? but this doesn't solve my issue, these parameters change absolutely nothing in behavior.

This error occurs for any request to this website, others seem to work fine. I also tried to set different user-agent header, but it doesn't help either. Website seem to work normaly when just opening from the browser.

• node.js
• web-scraping
• http-redirect
• node-fetch

• 1 It could be something in place to prevent scraping webpages? – somethinghere Commented 2 days ago
• @somethinghere Possibly, but what exactly could be there, it can't be ip because I can access from the browser, neither user-agent because I tried to use the same as in my browser. What else can it be? I also tried to use axios, and there is the same issue, but in Postman it works fine =( – dalvi Commented 2 days ago
• Well if we knew, scrape protection wouldn't be as good as it is. Cloudflare has robust systems in place, and I am not an engineer that worked on that at all, so I have no clue. Pretty sure a nodejs request will have something identifiable in its headers. – somethinghere Commented 2 days ago

1 Answer 1

So I finally figured it out! It happens to be that when reaching any of this site's url, in case you don't have their cookies, it attempts to set cookies and then redirect to the same url. Browsers (and postman) do handle this interaction with something called a cookie jar, if I am correct, but node-fetch and others seem to not to, so they need custom agent which implements this, I used this one and it has good examples npmjs/package/http-cookie-agent