[Common Issue] How can I optimize my Netlify build time?

If you are interested in learning more about how our bots approach building sites, these three articles are a great place to start:

So you might think, it’s great that Netlify builds every commit from my repos…but isn’t that a bit wasteful? The answer is, it depends, but we can at least give you some best practices for limiting waste. Maybe you’ll save a tree this month!

This article is in 3 parts: Optimizing what you build, Optimizing how you build, and Optimizing your site changes. Each offers opportunities to reduce build times or eliminate unnecessary builds.

Optimizing what you build

Netlify will happily build as many branches from your repository as you like - including known-broken branches, or branches that you don’t intend to browse or care if we build. So, our #1 recommendation to cut down on Build Minutes used is to check your configuration:

  1. Near the top of the Build & Deploy settings page for each site, you’ll see a card titled Deploy Contexts. In it you can choose which branches we build, and whether we build PR’s, or just commits. Consider switching off deploy previews for some branches if you do not follow a one-branch-per-feature model. There’s no wrong way to do build branches but we do suggest you consider the settings.
  2. You can tell us to skip any build by including the characters [skip ci] in your commit message.
  3. If you run local tests, complete your testing before committing. No need for us to rediscover what your unit tests or Jenkins server already knows and waste some CPU while doing it!
  4. If you know exactly when you want to build and are willing to trigger it yourself through our UI or via an incoming webhook, our Support team can enable a flag for your site(s) to skip all automatic builds from git, and use only the ones from your own custom webhooks which will still pull your code from git and perform a normal build.
  5. If you deploy manually using the CLI and have no intention of having us build, let us help you switch off continuous integration. It’s not possible to unlink by yourself, from our UI but once again you can comment in this thread with some site ID’s you’d like this configuration applied to. You can find your site ID on the General Settings page for the site.
  6. Be careful with your webhook configuration. Some folks have Contentful set up to save with every keystroke, and trigger a build too. Don’t do that :). Try to prevent unnecessary builds triggered via incoming webhook from services like your CMS or CI by configuring their auto-save or auto-commit features appropriately.

Optimizing how you build

There are as many tips and tricks for this as there are ways of building Netlify sites, but here are a few:

  1. Let us cache your dependencies. We automatically try to cache your gems and npm/yarn modules. We generally don’t reinstall them all unless you use the “clear build cache” checkbox in our UI when triggering a build. Note that any change to your package.json or yarn.lock or Gemfile.lock will result in us re-running the installation during next build, but it should still take advantange of the cache to some degree.
  2. Watch your dependencies. There are some packages that take a long time to install - cypress is a good example. If you don’t need some dependencies on Netlify, consider setting your $NODE_ENV environment variable to production and making them devDependencies.
  3. Not all build tools are created equal. Hugo is faster than most other build tools, for example. This isn’t to say you should port your site to a different static site generator, but if you are planning a large site and have not yet chosen a generator, consider build time as a factor.
  4. Configure your tools appropriately. Some SSGs use plugins that use a lot of memory and run slowly - they can even cause builds to fail with a timeout. This article, for example, has more details on some best practices for Gatsby + Netlify.

Optimizing your site changes

No matter how fast your build goes, we still have to store your files after build. If your build process changes many files with every deploy, every one of those files must be stored. If it instead changes only a single file, there is only one file to save, and one is always faster to save than many.

Look to optimize your build process so that it doesn’t change many unrelated files (or filenames) with each build. Changing fewer files with each build also allows your repeat visitors to browse your site faster, since they can leverage browser caching.

This article talks about the situation in depth, with some suggestions around how to avoid it.

Do you have other tips and tricks about how you’ve sped up your build? Share them with us below!

6 Likes

I would also point out this old thread that I just found - this person accidentally had multiple webhooks, so multiple builds were fired for no reason:

Making sure you don’t have redundant webhooks is something to pay attention to if you are trying to reduce build minutes used. :+1:

Is it possible to view this data in a similar way to how we view Netlify Analytics. If we could see how much minutes are being used per day in a 1 month period it would be really helpful.

Yesterday, I made some changes to my project decreasing build times from 10 minutes down to 4 minutes. I also modified some settings on netlify like disabling pull request deploy previews and such. Nonetheless, it will be really hard to understand how all these changes impact my build times if all I can see is a number for the entire month.

The data I was referring to:

If we had a chart i could tell after a week hey my total build times have decreased a lot or I still need to make more changes. I could tell if my build times decreased enough to re-enable a feature like deploy previews. One number for the total of a month isn’t enough in my opinion.

One of the best thing has been how this encourages us to create better projects by improving our build times. However, it’s really hard to determine how effective that change is over a period of time.

2 Likes

We’re working on it :wink: Something more granular should be coming soon and we’ll announce it in this thread.

2 Likes

Is it possible to enhance this a little bit please? This is a really good feature but I wish we could have a setting in the Dashboard where we could add our own keywords. In my opinion, this is going to be a massive impact on build times for projects like mine.

1. Dependabot
The bot makes quite a few PR into a project. If you have deploy previews enabled, your build times are going to be getting destroyed. Even when you have it disabled, merging these PRs can cause about 2-4 builds to be run. This certainly needs to be enhanced.

Today I had about 12 PR from the bot. I merged as many as i could as fast as I could until some of them had conflicts preventing merging. While waiting to have the bot resolve these conflicts netlify builds completed 1 build and skipped a few and began building the latest one. Once dependabot fixed the conflicts I merged them as well and once again netlify created a third build.

All this happened while I was rushing to merge them. Normally, I would just come back later and merge any remaining PRs. This is going to be causing anywhere between 2-4 builds for normal users. Something else that I have not yet considered is what happens when you have more than 1 build capacity. If i were to have 3 build capacity this might become a nightmare haha.


2. Netlify CMS
My project also incorporates netlify CMS and maintainers can write guides on the project. This includes uploading images and creating guides. The issue is that uploading a single image can trigger an entire build as well. This becomes really problematic even if you have not merged that PR because netlify CMS edits the repo directly with uploaded images.


If it was possible to somehow control what keywords to look for in the settings we could limit a lot of these builds from being triggered.

Great suggestion, but I don’t think we have immediate plans to implement it in the near term for the general use case of commits where you specify the commit message, which I understand is NOT your use case. I wonder if @erquhart could speak to the ability to customize the CMS commit message that the CMS uses?

I have nonetheless filed a feature request for it, and linked this thread to it so we can respond here in case we do.

One of our maintainers just opened an issue for this on the Netlify CMS repo, very doable.

1 Like

Can you check if the ignore setting in netlify.toml might work for you?

You could try something like this: git log -1 --pretty=%B | grep dependabot.
Git log has all kinds of info about a commit, so you could also check who committed.

2 Likes

Turns out custom commit messages are already available as a beta feature:

backend:
  commit_messages:
    create: 'Create {{collection}} “{{slug}}”'
    update: 'Update {{collection}} “{{slug}}”'
    delete: 'Delete {{collection}} “{{slug}}”'
    uploadMedia: '[skip ci] Upload “{{path}}”'
    deleteMedia: '[skip ci] Delete “{{path}}”'
2 Likes