I'm building a Nuxt 3 project. I need my build to generate a robots.txt file, just like this package states it does ->

After running "nuxt build" and/or "nuxt generate", the robots.txt does not appear in the output or public folders as I'd expect.

I'm definitely missing something and likely being an idiot here.. Does anyone know what I'm missing? Here's my code:

package.json

  "dependencies": {
    ...
    "@nuxtjs/robots": "^2.5.0", 
    }

nuxt.config.ts

 target: "static",
  runtimeConfig: {
    NUXT_STORYBLOK_PRODUCTION_KEY: process.env.NUXT_STORYBLOK_PRODUCTION_KEY,
    public: {
      CDN: process.env.CDN,
      NUXT_STORYBLOK_PREVIEW_KEY: process.env.NUXT_STORYBLOK_PREVIEW_KEY,
      NUXT_DOMAIN_NAME: process.env.NUXT_DOMAIN_NAME,
    },
  },
  modules: [
    ...
    "@nuxtjs/robots",
  ],
  robots: {
    UserAgent: "*",
    Disallow: "",
  },
}

I'm building a Nuxt 3 project. I need my build to generate a robots.txt file, just like this package states it does -> https://github./nuxt-munity/robots-module

After running "nuxt build" and/or "nuxt generate", the robots.txt does not appear in the output or public folders as I'd expect.

I'm definitely missing something and likely being an idiot here.. Does anyone know what I'm missing? Here's my code:

package.json

  "dependencies": {
    ...
    "@nuxtjs/robots": "^2.5.0", 
    }

nuxt.config.ts

 target: "static",
  runtimeConfig: {
    NUXT_STORYBLOK_PRODUCTION_KEY: process.env.NUXT_STORYBLOK_PRODUCTION_KEY,
    public: {
      CDN: process.env.CDN,
      NUXT_STORYBLOK_PREVIEW_KEY: process.env.NUXT_STORYBLOK_PREVIEW_KEY,
      NUXT_DOMAIN_NAME: process.env.NUXT_DOMAIN_NAME,
    },
  },
  modules: [
    ...
    "@nuxtjs/robots",
  ],
  robots: {
    UserAgent: "*",
    Disallow: "",
  },
}

• javascript
• nuxt.js
• robots.txt
• nuxt3.js

• It should be generate and available in dist if I'm not mistaken. Otherwise you can always run preview and inspect the generated payload to double check. – kissu Commented Nov 1, 2022 at 10:34
• Nvm, it's the .output directory rather. – kissu Commented Nov 1, 2022 at 10:39
• It doesn't appear for me in output. If it should appear there and I get no other responses on this question, it's likely I have a dependency conflict with another package and it may be a case of stripping down the project until I find the problem. – Evan Meredith-Davies Commented Nov 1, 2022 at 10:57
• Tried to preview it? I doubt there is a conflict tbh. – kissu Commented Nov 1, 2022 at 11:01
• Yep. Stil nothing. Build / Generate && Preview, tried it all. No Robots file gets generated in that output folder or any of it's subfolders. – Evan Meredith-Davies Commented Nov 1, 2022 at 11:25

7 Answers 7

You are using an inpatible version of nuxt robots for your project. Ensure you are using version ^3.0.0. Any versions below this are not patible with nuxt 3.

npm install @nuxtjs/[email protected]  

Then ensure your nuxt.config.ts looks similar to below. As John Overstreet mentioned above the simple method appears flawed.

I created a config folder in the root directory of my project and placed my robots.txt file within it:

export default {
  modules: [
    ['@nuxtjs/robots', { configPath: "~/config/robots.config" }]
  ]
}

You will also need to delete the robots.txt file from your public folder

Build/Generate your project and go to '/robots.txt'

The file is generated dynamically upon request of the above route.

With mine and John's help above I hope this helps resolve your issues :)

I just successfully installed this module on Nuxt 3, and I think there are a couple of things to note here.

The first is that I've never managed to get the module options for any module to work in Nuxt 3 the way you have shown (top-level options, according to the module docs):

  modules: [
    ...
    "@nuxtjs/robots",
  ],
  robots: {
    UserAgent: "*",
    Disallow: "",
  }

Have you tried the other options? You can also try using the Robots Config file, or pass the options when declaring the module (from the README.md for the repo):

export default {
  modules: [
    // Simple usage
    '@nuxtjs/robots',

    // With options
    ['@nuxtjs/robots', { /* module options */ }]
  ]
}

The other thing is that I also do not see a generated robots.txt file anywhere after running the build or dev, but if I go to '/robots.txt' in the dev or build preview, I can see the output working as intended. Have you tried visiting the path?

It looks like something the server generates when the route is visited rather than a static file generated on build by default. I think you can change that in the options, but it's not something I need so I haven't dug into it.

Hopefully, that helps somewhat!

Hi i think to be iso with the Nuxt 3 ecosystem it will be better to use this module:

nuxt-simple-robots

You can install it through yarn add -D nuxt-simple-robots

(In the same page you can find the modules for Sitemap as well and all SEO needed modules for Nuxt 3 SEO)

Here is the documentation about how to install it and work with it

https://nuxtseo./robots/getting-started/installation

On Nuxt 3 this generates the default robots.txt file in dist, but setting the described parameters doesn't do anything (inline or not).

I manually modify my dist/robots.txt file before deployment.

Not ideal, but at least it generates the default file.

It didn't work for me either. I tried all the methods.

1. As per the documentation with npm install @nuxtjs/robots.
2. I tried with version npm install @nuxtjs/[email protected] and creating a robots.config file.

Both didn't work. Finally, I ended up creating a robots.txt file in the public folder.

In my case I just need to delete my /public/robots.txt, retain the /public/_robots.txt, and it started working.

For Nuxt 3

Step-1 Install

npm install @nuxtjs/[email protected]  

Step-2 Add below in nuxt.config.js

modules: [
[ '@nuxtjs/robots',
    {
        rules :{
            UserAgent: '*',
            Disallow: '/'
           }
     }
] ]

Step-3 Build/Generate your project

You will see dynamically generated robots.mjs file in .nuxt folder with given rules.

Now you can check robots.txt at - https://<your domain>/robots.txt OR http://localhost:3000/robots.txt