Builds always fail until clearing cache and re-trying

Recently, my netlify builds on a particular project started failing with a nondescript error:

8:18:45 PM: Finished saving go dependencies
8:18:45 PM: Error running command: Build script returned non-zero exit code: 1

But when I manually “clear cache and retry”, it builds just fine. It also runs just fine locally on various machines (windows/mac/linux).

For example, here is one that failed:
https://app.netlify.com/sites/adage-frontend/deploys/5e852ef33473ae0006386745

And the manual retry was fine:
https://app.netlify.com/sites/adage-frontend/deploys/5e852f81b140ab00118ce435

I figured it started happening because I recently added some more dependencies that took longer to fetch and install, and just made the deploy time-out. But I’m not sure. Does anyone have any insight as to why this could be happening? Is there a way to increase the build timeout?

Possibly related: Build failures on every push except pass on cache rebuild

However I’ve been frequently deleting node_modules and yarn.lock and reinstalling, locally, before committing. So I don’t think my yarn.lock is the problem.

EDIT: Also possibly related, a very recent issue on Create React App (which I am using for this project): https://github.com/facebook/create-react-app/issues/8751

hi there, did you see this guide already?

there might be something in there. as far as the build error goes, there may be something useful higher up in the log. if you can’t fix it, please do paste the whole deploy log, thanks!

Sorry for the late response.

I’ve looked through that guide, and the only possible problem is #2: Your dependencies have binary components, which need to be rebuilt for some reason. Here are my dependencies:

{
  "dependencies": {
    "@greenelab/hclust": "^0.0.0",
    "@stdlib/stdlib": "^0.0.91",
    "color": "^3.1.2",
    "d3": "^5.15.0",
    "history": "^4.10.1",
    "immer": "^6.0.2",
    "intersection-observer": "^0.7.0",
    "multtest": "^0.1.7",
    "object-sizeof": "^1.5.3",
    "prop-types": "^15.7.2",
    "rc-slider": "^9.2.4",
    "react": "^16.13.1",
    "react-dom": "^16.13.1",
    "react-intersection-observer": "^8.26.1",
    "react-linkify": "^1.0.0-alpha",
    "react-redux": "^7.2.0",
    "react-router-dom": "^5.1.2",
    "react-scripts": "^3.4.1",
    "react-transition-group": "^4.3.0",
    "redux": "^4.0.5",
    "redux-actions": "^2.6.5",
    "redux-logger": "^3.0.6",
    "redux-query-sync": "^0.1.9",
    "redux-sequence-action": "^0.2.1",
    "redux-thunk": "^2.3.0",
    "unescape": "^1.0.1",
    "use-debounce": "^3.4.0",
    "workerize-loader": "^1.1.0"
  },
  "devDependencies": {
    "@testing-library/jest-dom": "^5.3.0",
    "@testing-library/react": "^10.0.2",
    "eslint-config-greenelab": "git+https://git@github.com/greenelab/eslint-config-greenelab.git",
    "node-fetch": "^2.6.0",
    "prettier-config-greenelab": "git+https://git@github.com/greenelab/prettier-config-greenelab.git",
    "rewire": "^5.0.0"
  }
}

Here is the build log:

5:52:51 PM: Build ready to start
5:52:54 PM: build-image version: 2dbd444fcdce00cf06325060a8238d5ae3e86774
5:52:54 PM: build-image tag: v3.3.7
5:52:54 PM: buildbot version: 7f867b365e4c3934ca5b72b9d51f0a4d1f96685d
5:52:54 PM: Fetching cached dependencies
5:52:54 PM: Starting to download cache of 362.7MB
5:53:01 PM: Finished downloading cache in 7.402903356s
5:53:01 PM: Starting to extract cache
5:53:20 PM: Finished extracting cache in 19.035413942s
5:53:21 PM: Finished fetching cache in 26.575510656s
5:53:21 PM: Starting to prepare the repo for build
5:53:21 PM: Preparing Git Reference pull/153/head
5:53:22 PM: Starting build script
5:53:22 PM: Installing dependencies
5:53:23 PM: Started restoring cached node version
5:53:29 PM: Finished restoring cached node version
5:53:30 PM: Downloading and installing node v10.20.0...
5:53:30 PM: Downloading https://nodejs.org/dist/v10.20.0/node-v10.20.0-linux-x64.tar.xz...
5:53:30 PM: 
#######
5:53:30 PM:                               10.5%
5:53:31 PM: 
############################
5:53:31 PM: ############################################ 100.0%
5:53:31 PM: Computing checksum with sha256sum
5:53:31 PM: Checksums matched!
5:53:35 PM: Now using node v10.20.0 (npm v6.14.4)
5:53:35 PM: Attempting ruby version 2.6.2, read from environment
5:53:37 PM: Using ruby version 2.6.2
5:53:37 PM: Using PHP version 5.6
5:53:37 PM: Started restoring cached node modules
5:53:37 PM: Finished restoring cached node modules
5:53:37 PM: Started restoring cached yarn cache
5:53:37 PM: Finished restoring cached yarn cache
5:53:38 PM: Installing NPM modules using Yarn version 1.17.0
5:53:39 PM: yarn install v1.17.0
5:53:39 PM: warning package.json: No license field
5:53:39 PM: warning adage-frontend@0.2.8: No license field
5:53:39 PM: [1/4] Resolving packages...
5:53:40 PM: [2/4] Fetching packages...
5:55:17 PM: info fsevents@2.1.2: The platform "linux" is incompatible with this module.
5:55:17 PM: info "fsevents@2.1.2" is an optional dependency and failed compatibility check. Excluding it from installation.
5:55:17 PM: info fsevents@1.2.12: The platform "linux" is incompatible with this module.
5:55:17 PM: info "fsevents@1.2.12" is an optional dependency and failed compatibility check. Excluding it from installation.
5:55:17 PM: [3/4] Linking dependencies...
5:55:17 PM: warning "react-scripts > @typescript-eslint/eslint-plugin > tsutils@3.17.1" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
5:55:17 PM: warning " > workerize-loader@1.1.0" has unmet peer dependency "webpack@*".
5:55:49 PM: [4/4] Building fresh packages...
5:55:58 PM: success Saved lockfile.
5:55:58 PM: Done in 138.62s.
5:55:58 PM: NPM modules installed using Yarn
5:55:58 PM: warning package.json: No license field
5:55:58 PM: Started restoring cached go cache
5:55:58 PM: Finished restoring cached go cache
5:55:58 PM: unset GOOS;
5:55:58 PM: unset GOARCH;
5:55:58 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.12.linux.amd64';
5:55:58 PM: export PATH="/opt/buildhome/.gimme/versions/go1.12.linux.amd64/bin:${PATH}";
5:55:58 PM: go version >&2;
5:55:58 PM: export GIMME_ENV='/opt/buildhome/.gimme/env/go1.12.linux.amd64.env';
5:55:58 PM: go version go1.12 linux/amd64
5:55:58 PM: Installing missing commands
5:55:58 PM: Verify run directory
5:55:58 PM: Executing user command: yarn build
5:55:59 PM: yarn run v1.17.0
5:55:59 PM: warning package.json: No license field
5:55:59 PM: $ node ./scripts/build.js
5:56:00 PM: Creating an optimized production build...
5:56:04 PM: Failed to compile.
5:56:04 PM: ./src/index.js
**5:56:04 PM: Error: [BABEL] /opt/build/repo/src/index.js: Cannot find module '@babel/helper-call-delegate' (While processing: "/opt/build/repo/node_modules/babel-preset-react-app/index.js")**
5:56:04 PM: error Command failed with exit code 1.
5:56:04 PM: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
5:56:04 PM: Skipping functions preparation step: no functions directory set
5:56:04 PM: Caching artifacts
5:56:04 PM: Started saving node modules
5:56:04 PM: Finished saving node modules
5:56:04 PM: Started saving yarn cache
5:56:04 PM: Finished saving yarn cache
5:56:04 PM: Started saving pip cache
5:56:04 PM: Finished saving pip cache
5:56:04 PM: Started saving emacs cask dependencies
5:56:04 PM: Finished saving emacs cask dependencies
5:56:04 PM: Started saving maven dependencies
5:56:04 PM: Finished saving maven dependencies
5:56:04 PM: Started saving boot dependencies
5:56:04 PM: Finished saving boot dependencies
5:56:04 PM: Started saving go dependencies
5:56:04 PM: Finished saving go dependencies
5:56:14 PM: Error running command: Build script returned non-zero exit code: 1
5:56:14 PM: Failing build: Failed to build site
5:56:14 PM: failed during stage 'building site': Build script returned non-zero exit code: 1
5:56:14 PM: Finished processing build request in 3m20.621889978s

Is there a way to simply always clear the cache before building? Perhaps this post’s suggestion of rm -rf $NETLIFY_CACHE_DIR && run-my-build still works?

hi there, I am not sure if that command would work still off the top of my head - but I recommend you try it and let us know?

The specific command rm -rf $NETLIFY_CACHE_DIR && run-my-build didn’t work. I don’t think $NETLIFY_CACHE_DIR is set as an environment variable (but it would be good if it was!).

I also tried a bunch of other things for several hours, but had to stop because I was wasting too much time on it. It’s been very frustrating.

I tried removing /opt/cache/node_modules and other related directories in the cache directory before the build command. But then it occurred to me that Netlify must be doing installation steps (instaslling node, npm install, etc) even before my build commands run, and that’s where the error is happening. So I would need a way to preempt that.

Perhaps the new build plugins would be the idiomatic solution here. But I really wish there was a way to always clear the cache – always start from a clean slate to ensure a proper build – either through a checkbox in the UI, or maybe an environment variable flag.

1 Like

Netlify’s caching and API calls are busted since this latest update. Lots of deploys that used to work now failing, not calling backend properly, etc. Not even Netlify identity is working. You are right, it is incredibly frustrating.

@appdevnick - sorry to hear you are having so much trouble. That isn’t what we want, of course, and I agree that that sounds frustrating.

Can you tell me a little more information about what isn’t working? do you have a deploy/build log with some errors we can look at? What isn’t working specifically about identity?

at the very least your netlify site name would be helpful! thanks.

1 Like

Hi perry, thank you for replying. I posted a few days ago already with a full rundown of my issue, but no replies so far.

So as not to hijack this thread, here’s my thread: Vue (Gridsome SSG) site all of a sudden not loading any API content after recent Netlify changes

I will go back and update it with anything relevant if I can think of it. Thank you for your assistance, I appreciate your time.

@vincerubinetti this issue has been experienced by create-react-app users on other systems like GitLab CI as well: https://github.com/facebook/create-react-app/issues/8751

I don’t see any reason that this would be a Netlify specific problem - I’d recommend reaching out to the create-react-app community for help with code itself.

I finally managed to solve this in a kludge-like way: I created one of the new build plugins to clear the yarn cache before each install.

In ./netlify/plugin/index.js:

const util = require('util');
const exec = util.promisify(require('child_process').exec);

module.exports = {
  async onInit() {
    console.log('deleting node_modules');
    await exec('rm -rf node_modules');
    console.log('clearing yarn cache');
    await exec('yarn cache clean');
    console.log('running yarn install');
    await exec('yarn install');
  }
};

In .neltlify/plugin/manifest.yml:

name: netlify-plugin-clear-cache

In netlify.toml:

[[plugins]]
package = "./.netlify/plugin"

All this does is clear the node_modules directory, clear the yarn cache, and reinstall all the yarn packages. Netlify is already installing yarn packages before this step, so this will at least double the time it takes to make pull request deploy previews, for example. But at least I don’t have to keep manually clicking “clear cache and redeploy”, and sometimes forgetting, and ending up with red x’s all over my github commit history.

I suppose you could accomplish the same thing by just adding the yarn cache clean and etc. commands to your npm run build script.

I REALLY wish Netlify just gave me an option to clear the yarn/npm cach, either in netlify.toml or in the online UI.

A recent, related issue:

1 Like

It seems like this issue might ultimately be due to this issue:

I think one of my dependencies stdlib is a monorepo, and I believe it also compiles to binary or web assembly or something, which the guide said might be a problem because it doesn’t get re-built. I’ve frequently had problems with it running correctly locally until I clear the yarn cache and reinstall. And although my memory is hazy, I think the Netlify pull request previews started failing around the same time I added this library to my project.

Hey @vincerubinetti,

We really appreciate the deep dive and feedback you’ve provided. Leaving your findings here for posterity is truly appreciated :handshake:!