[Support Guide] Using private NPM modules on Netlify

OK, thanks for talking through that with us and trying all the things. I’d post a link to this thread and description of your problem and hoped for solution here as a feature request:

I do think that the advice will probably be “WONTFIX but see the build plugins beta where you can implement this yourself” (in a couple of months), but instead of us playing telephone to our build system developers probably better to let you represent yourself (and see any responses they send to you :))

Let me know if you can’t do that for some reason and I can do it for you and will still link you to the open issue to track follow-ups there.

@radu any chance you still have a .npmrc file checked into your git repo? I am unable to reproduce your issue with our script, but having an existing .npmrc file inside your repo might be the reason you are having this problem.

Another option would be to replace the check if a .npmrc file exists with a new environment variable which is set when spawning yarn from a child process. In that case the script would look something like this:

const fs = require('fs')
const { spawnSync } = require('child_process')

// Netlify does not support Github Packages (or other private package registries besides npm), options are:
//   - Commit .npmrc to repo - However, now we have a secret token inside our repo
//   - Environment variable in .npmrc - However, this requires all developer machines to have the same environment variable configured
//   - Get creative with the preinstall script... :)

// Only run this script on Netlify
if (process.env.NETLIFY === 'true') { // this is a default Netlify environment variable
	// Check if .npmrc was already generated by this script. If it does then do nothing (otherwise we create an infinite yarn loop)
	if (process.env.NETLIFY_NPMRC_DONE !== 'true') {
		// Create .npmrc
		fs.writeFileSync('.npmrc', `//npm.pkg.github.com/:_authToken=${process.env.GITHUB_TOKEN}\n@oliverit:registry=https://npm.pkg.github.com/\n`)
		fs.chmodSync('.npmrc', 0o600)
		// Run yarn again, because the yarn process which is executing
		// this script won't pick up the .npmrc file we just created.
		// The original yarn process will continue after this second yarn process finishes,
		// and when it does it will report "success Already up-to-date."
		spawnSync('yarn', { stdio: 'inherit', env: { ...process.env, NETLIFY_NPMRC_DONE: true } })
	}
}
2 Likes

@rvanmil :tada: thanks for the solution it’s working! Was thinking about doing something similar, but didn’t know I can send new env vars to spawn - that was the missing piece of the puzzle.

Regarding your question - no, I don’t have a checked-in npmrc file - that’s what I wanted to avoid.

@fool as this is now fixed, and since I just got access to the beta build plugins ( thanks Netlify team :heart:), I will try and experiment with the new build system, as I’m convinced I can solve the problem much more elegantly with that.

4 Likes

This is solution is a non-starter for us. Committing a local npmrc file breaks a ton of ecosystem workflows and requiring that everyone on the team have their token in an env var is an unreasonable pain point, and not how the ecosystem works generally. If Netlify wants to continue the (nice) experience of auto installing deps then it should automatically used the configured NPM_TOKEN when installing, or allow users to override at run time

2 Likes

Sorry to hear you don’t like the available solution. It’s working for a lot of folks, so probably not a high priority for our team to change things right now, but would you mind outlining your perfect solution here so the wider team can consider it?

Just wanted to share my experience in case it helps anyone in the future.

After a day of banging my head against the wall I managed to get my private module working with Netlify by adding curly brackets around the token in .npmrc as outlined in the original post, eg.

authToken=${FONTAWESOME_NPM_AUTH_TOKEN}

Unfortunately the curly brackets prevent yarn from accessing the env variable when working locally, bit of a pain but I just need to remember to add them in before any commits.

Thanks for the thread!

edit: on the plus side I’ve spent so much time reading docs and blogs I now know considerably more about using font awesome with react

1 Like

damn! Thank you so much for sharing that tip!!

I can confirm that this method works perfectly for me. Adding a .npmrc to my repo was not an option for me and my team so am really happy this works. Thanks @rvanmil

2 Likes

According to official doc of npmrc and npm config, could NPM_CONFIG_USERCONFIG be the solution?

This is how this issue is solved in github actions with this action in the marketplace. It writes .npmrc content to a temp file and tells npm to use it through NPM_CONFIG_USERCONFIG.

Then entire process would be:

  1. run a custom node.js script to do whatever, like setting environment variables (e.g. NPM_CONFIG_USERCONFIG) or creating files (e.g. /opt/build/repo/.npmrc)
  2. run npm install
1 Like

Feel free to check out this workaround: https://github.com/MoYummy/netlify-demo. :slight_smile:

2 Likes

Thanks so much for that demo, @MoYummy! @mpan-wework I think it will do what you are asking :slight_smile:

Maybe something like this in package.json?

"preinstall": "if [ ! -z \"$NETLIFY\" ]; then echo \"//npm.pkg.github.com/:_authToken=${NPM_TOKEN}\" >~/.npmrc; fi || true",

Thanks for sharing that @loof and welcome to the Netlify community site! :slight_smile:

@fool perhaps now is a good time to add built in support for at least private packages on GitHub? It looks like a lot of folks will be moving their private packages that way soon https://github.blog/2020-03-16-npm-is-joining-github/

This currently doesn’t seem to be possible with build plugins (beta). Related issue over here: https://github.com/netlify/build/issues/1149

This is still an absolute chore with netlify. With any other system I can just authorize npm with a token, or have a special build-script, etc.

Why is the solution for private npm on netlify a workaround? It’s an exceptionally common use-case.

My ultimate suggestion is that you just have support for a build-netlify task, which, if it exists, is run by netlify instead of the regular build task. This would make this process a lot easier.

Seems pretty straightforward to me - set an environment variable and use it. Could you explain in more detail what’s so tedious about this 1 line config?

1 Like

Hi team, we’re in the same situation as many of you in that we’re not comfortable editing our .npmrc file locally. The solution that we’ve got working is to edit it as part of the “Build command”
Steps:

  1. Add NPM_TOKEN to environment variables in netlify
  2. Disable netlify’s install - we achieved this by adding the NPM_FLAGS = --dry-run environment variable
  3. Edit .npmrc - Update your build command to:
    echo '//npm.pkg.github.com/:_authToken=${NPM_TOKEN}' >> .npmrc && npm install && npm run build

Hopefully this helps!

1 Like

This has already been explained multiple times in this thread… :thinking:

None of the solutions given are really acceptable.

Directly adding the .npmrc file causes local npm installs to fail due to missing the variable ie: //npm.pkg.github.com/:_authToken=${NPM_TOKEN}. Given that developers normally login to npm using the npm login command locally, there should be absolutely no reason for them to also need to set NPM_TOKEN on their machine just so that npm install can work on their machine.

Using all these workarounds with dry-run means that we get billed more than we would have without the dry run. I think that speaks for itself as to why it’s not a viable long term solution.

And we shouldn’t have to be maintaining a custom preinstall hook in our package.json for this. This is really something that should be handled by the netlify.toml file, such as a “preinstall_command” hook.

After diving into this issue a bit, I found this. Adding NPM_TOKEN as an environment automatically adds the registry information for NPM. Is it possible to have this functionality added for github as well, such as GITHUB_TOKEN?

Ideally there should be a pre-install hook for build commands.

2 Likes