`npm install` works on build image but fails in production

My Netlify instance name is omtone-admin.netlify.com.

I’m trying to set up CI / CD on Netlify for a new project. The project is a SPA with its source code hosted on Bitbucket and uses parcel-bundler. I have a build script that works locally. However, when Netlify goes to deploy my site it fails to install any NPM dependencies. Here’s the first piece of the build log (the same issues go on for a long, long time).

3:15:33 PM: Build ready to start
3:15:34 PM: build-image version: 9cade8af58c2cf3a17a1e9433d2e979149488837
3:15:34 PM: build-image tag: v3.3.5
3:15:34 PM: buildbot version: 2fd8f04962d5dbc77b79b9c2f5ab7a077e87376b
3:15:34 PM: Fetching cached dependencies
3:15:34 PM: Starting to download cache of 254.9KB
3:15:34 PM: Finished downloading cache in 106.079689ms
3:15:34 PM: Starting to extract cache
3:15:35 PM: Failed to fetch cache, continuing with build
3:15:35 PM: Starting to prepare the repo for build
3:15:35 PM: No cached dependencies found. Cloning fresh repo
3:15:35 PM: git clone git@bitbucket.org:omtone/admin.omtone.life
3:15:37 PM: Preparing Git Reference refs/heads/production
3:15:37 PM: Starting build script
3:15:37 PM: Installing dependencies
3:15:39 PM: Downloading and installing node v10.19.0...
3:15:39 PM: Downloading https://nodejs.org/dist/v10.19.0/node-v10.19.0-linux-x64.tar.xz...
3:15:39 PM: 
###########
3:15:39 PM:  15.9%
3:15:39 PM: 
######################################################
3:15:39 PM: ################## 100.0%
3:15:40 PM: Computing checksum with sha256sum
3:15:40 PM: Checksums matched!
3:15:43 PM: Now using node v10.19.0 (npm v6.13.4)
3:15:43 PM: Attempting ruby version 2.6.2, read from environment
3:15:44 PM: Using ruby version 2.6.2
3:15:45 PM: Using PHP version 5.6
3:15:45 PM: Started restoring cached node modules
3:15:45 PM: Finished restoring cached node modules
3:15:45 PM: Installing NPM modules using NPM version 6.13.4
3:15:53 PM: npm
3:15:53 PM: WARN tar ENOENT: no such file or directory, open '/opt/build/repo/node_modules/.staging/stream-http-1abcc9dc/package.json'
3:15:53 PM: npm
3:15:53 PM: WARN tar ENOENT: no such file or directory, open '/opt/build/repo/node_modules/.staging/node-libs-browser-804a6c1e/package.json'

Like I said, this goes on for a while. Here’s the end of the log from another failed build:

4:04:23 PM: Error during NPM install
4:04:23 PM: Error running command: Build script returned non-zero exit code: 1
4:04:23 PM: Failing build: Failed to build site
4:04:23 PM: failed during stage 'building site': Build script returned non-zero exit code: 1
4:04:23 PM: Finished processing build request in 40.317855024s

Looking at other cases, I ran across the local build image. I followed the instructions to test the build locally and didn’t have any issues pulling NPM dependencies as you can see from the truncated log:

buildbot@4bb8c8fd1156:/$ build npm run build
Installing dependencies
v10.19.0 is already installed.
Now using node v10.19.0 (npm v6.13.4)
Attempting ruby version 2.6.2, read from environment
Using ruby version 2.6.2
Using PHP version 5.6
Started restoring cached node modules
Finished restoring cached node modules
Installing NPM modules using NPM version 6.13.4
mkdir: cannot create directory ‘/opt/buildhome/tmp’: File exists
The authenticity of host 'bitbucket.org (18.205.93.0)' can't be established.9.0
RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting (yes/no)? [        ..........] \ extract:babel-preset-react-app: sill extract yargs-parser@13[            ......] \ extract:@material-ui/icons: sill extract @material-ui/icons@4.5.1 extracted to /opt/buildhome/repo/node_modules/.s

At this point, I’m kind of stumped. It seems like Netlify’s production environment is behaving differently than the local environment. I can work around the issue by building locally and committing the dist directory to my production branch, but I’d rather not clutter my repo with transpiled code if I don’t have to. I’d be grateful for any suggestions.

Thanks in advance!

Chris.

Whenever we see an npm failure around a path containing /.staging/ we know one of two things is wrong:

  1. either you have committed your node_modules to git and that won’t work well in our build environment
  2. or you need to regenerate package-lock.json as it has some bad state in it.

Could you let me know if either of those fixes help?

No luck on either front. I made sure to add node_modules to .gitignore at the beginning of the project. Deleting and regenerating package-lock.json didn’t build with a different outcome either. I’m confused why the build would be successful using the local build image and unsuccessful in production.

can you try clearing your deploy cache and trying again? Also, could you provide a link to the git repo you are trying to build from?

The repo is private on Bitbucket, but I’d be happy to add a reviewer if you provide me with an email address… Clearing the cache and rebuilding wasn’t successful, rendering the same results.

one other thing we might try - can you reproduce the error on a different site (by creating a new repo, uploading the same files, and seeing if this also causes a problem)?

just be absolutely sure that the repo does not contain the node_modules folder and also no package-lock file.

I deleted package-lock.json and rebuilt. It started installing dependencies and then threw on an import I forgot about from another private repo. Maybe that’s where the issue has been coming from. I’m struggling to add SSH keys to my account so the build server can pull the remote repo. Where can I find SSH key management on my account?

There is no “management UI” for those. There’s an API endpoint but its usage is fraught with peril, so instead, please read this article:

and based on its advice, figure out what you’ll need (either changing submodules to public so you have at most 0 private submodules,or instead creating a “dummy” user to hold the deploy key as an access key), and let us know what decision you come to - if we end up creating a new key for you builds will IMMEDIATELY stop working until you replace with the one we’ve made - and we’ll help out and generate a new key for you.

Opening the repo up as public has resolved the issue. Thanks for working with me through my build issues.

great. glad its working!