[Support Guide] Using private NPM modules on Netlify

@Scott that was me that opened that :slight_smile:

I was unfortunately unable to get the workaround involving --dry-run to work, which while inefficient seemed like the best way to get everything working without any custom scripts. The dry run was still attempting to contact the registry and check the packages on the first (uncached) request, which would fail the build.

We’ve instead opted for manually setting the token as a local environment variable on our developers machines for the time being.

Thanks for the update though, and the refund. I understand this is a fairly new and niche problem right now, so I’d also like to voice my support for adding custom pre-install commands to the build pipeline. I imagine that is something that’s a lot more useful to a lot more people anyways, and would help to easily fix this as well!

Hey @logo,

Makes sense! I appreciate your understanding and, although it’s not the working method we’d like, I’m glad that you have a solution which is enough to keep you going :+1:. We’ll be sure to feed back both in the linked issue and here if there are any developments.

1 Like

I am having similar issues with the pre-subscribed solutions as @logo. Netlify should expose functionality that can be commonly found in CI/CD pipelines (like CircleCI) for building assets from private repos. Not all people use NPM or Github for hosting private assets, my company uses artifactory. Having a local .npmrc file breaks local development for us, and it has been challenging to find a solution that allows us to pull assets from our private repository.

Here is a sample snippet of code from a config.yml file that we use to set these variables during a CI/CD build.

commands:
    configure_npm:
        description: "Configure the npm client to use private artifactory as registry"
        steps:
          - run: npm config set _auth $NPM_AUTH
          - run: npm config set email $NPM_EMAIL
          - run: npm config set always-auth true
          - run: npm config set registry https://website.com/artifactory/api/npm/reponame/

Couldn’t the netlify.toml file expose similar type steps to make the build phase easier to configure?

Good news, a fix for this issue has been put on the Netlify roadmap :smiley:
Refer to these GitHub issues:

This is genius and works like a charm!

Wanted to say that @mpan-wework’s suggestion worked for me, and it felt like the least amount of compromise. The tricky bit is that the developers on my team need to use both private npm packages and packages published to a self-hosted GitLab registry, so there are multiple config steps required. I was able to create an .npmrc file in an arbitrary folder within my project(s), which is setup to use environment variables that I’ve set at a team level. I then added a line to netlify.toml to set the NPM_CONFIG_USERCONFIG option.

Inside netlify.toml:

[build.environment]
  NPM_CONFIG_USERCONFIG = "./netlify/.npmrc"

And then I have a file within my project at ./netlify/.npmrc:

# Auth for private GitLab registry modules
@my-scope:registry=https://${gitlab_instance_url}/api/v4/packages/npm/
//${gitlab_instance_url}/api/v4/packages/npm/:_authToken=${GL_REGISTRY_TOKEN}

# Auth for private npm modules
//registry.npmjs.org/:_authToken=${NPM_TOKEN}

The variables NPM_TOKEN and GL_REGISTRY_TOKEN are both exposed in the Team / Sites / Global site settings.

The downside: every project we build needs to include this file and this configuration in netlify.toml, but this is fairly minor.

The upside: everything “just works” without forcing developers to alter their local development environments to accommodate builds in Netlify. This file is only used in the context of Netlify, and it only needs to be updated if and when we add more registries.

7 Likes

Thank you so much for taking the time to write this up in such detail, @grahaml!

1 Like

@grahaml thanks for sharing such a beautiful solution!

This is indeed the best solution, as it allows developers to have their own git-ignored .npmrc in the main folder with the tokens set directly in the file (so no export or set env variable script has to be called) and to leverage the env variables in Netlify properly. Thank you so much!

Does NOT work with YARN throws error like it completely ignores npmrc.

Could not find package “@scope/package” in the NPM registry
info Visit yarn install | Yarn for documentation about this command.
Error during Yarn install

It seems like it doesn’t understand .npmrc file set in the netlify.toml

How do you set authentication in yarnrc instead?

"@scope:registry" "https://npm.pkg.github.com"

Hey @FlatFlowerFrogFlag, not my area of expertise though there appears to be some workarounds over in the yarn repo :slight_smile:

Took a lot of digging but eventually came across this issue which pointed me to this line which shows that

If you only need to set an auth token to access private NPM modules, you only need to set the NPM_TOKEN env variable.

You do not need to create a .npmrc file in your repo as the npm build process will detect the env var and do it for you. This will not help you if you also need to use a different registry, but my guess is most folks just need the token set.

I hope the netlify team will update the instructions and point this out! It would save a lot of folks some hassle.

1 Like

@theo, that’s great sleuthing! We’ll get this Support Guide updated to reflect that. Thank you!!

I tried using @rvanmil’s preinstall approach with yarn 1.22.4 and Github Package Registry but it did not work for me. Later I found this Yarn seems to do use the right registry token (GitHub npm private registry) · Issue #8015 · yarnpkg/yarn · GitHub which indicated that the token must also be passed in an authorization header. I’ve modified his script to the following and it appears to work:

// Netlify does not support Github Packages (or other private package registries besides npm), options are:
//   - Commit .npmrc to repo - However, now we have a secret token inside our repo
//   - Environment variable in .npmrc - However, this requires all developer machines to have the same environment variable configured
//   - Get creative with the preinstall script... :)

const fs = require('fs');
const { spawnSync } = require('child_process');

const scope = 'your_scope';
const authToken = process.env.GITHUB_TOKEN;

// Only run this script on Netlify, this is a default Netlify environment variable
if (process.env.NETLIFY === 'true') {
  // Check if this has already run (we spawn yarn again at the end)
  if (process.env.NETLIFY_NPMRC_DONE === 'true') {
    return;
  }

  // Check if .npmrc already exists, if it does then do nothing (otherwise we create an infinite yarn loop)
  if (fs.existsSync('.npmrc')) {
    console.warn('Skipping .npmrc write because it already exists');
    return;
  }

  console.log(`Creating .npmrc with auth tokens for @${scope}, provided by GITHUB_TOKEN environment variable...`);

  // Create .npmrc
  fs.writeFileSync(
    '.npmrc',
    `//npm.pkg.github.com/:_authToken=${authToken}\n//npm.pkg.github.com/:_header:Authorization=token ${authToken}\n@${scope}:registry=https://npm.pkg.github.com\n`,
  );

  // Is this necessary?
  fs.chmodSync('.npmrc', 0o600);

  // Run yarn again, because the yarn process which is executing
  // this script won't pick up the .npmrc file we just created.
  // The original yarn process will continue after this second yarn process finishes,
  // and when it does it will report "success Already up-to-date."
  console.log(`.npmrc created. Starting yarn install...`);
  spawnSync('yarn', { stdio: 'inherit', env: { ...process.env, NETLIFY_NPMRC_DONE: true } });
}

I use neither .yarnrc nor .npmrc in the project.

Hiya @bdefore and welcome to our community! Thanks so much for that great workflow suggestion and for sharing your code. Just to clarify, could you confirm if you do have things working acceptably (I understand it is rather indirect, but you’re not blocked, right?)

Yes, this is working sufficiently for me, although I do wish it weren’t necessary and there was a more integrated solution through toml configuration. This approach only satisfies because I don’t otherwise use .npmrc for configuration.

One addition I’ve made since posting this is to check for YARN_VERSION and log out that it has no effect if using yarn 2 or higher - this version of yarn no longer infers configuration from .npmrc.

Ran into an issue while trying to use AWS Codeartifact. The problem here is that the tokens need to be generated at build time, so you can’t add a setting to the environment as its not possible to have long live tokens like you might for GitHub or some other private package repos.

In the end --dry-run worked best for my use case. Skipping the install meant could then run the setup script and then run a new install with the correct config.