How to parallelize Netlify builds to make them faster

At Fresh Ideas we want to be able to use the same repository to power a lot of sites. We have managed to do that with freshinabox.co.zw, freshmarket.co.zw, bottlestore.co.zw, freshfarm.co.zw and some others.

This has been incredibly helpful since before we had separate apps for each. Now the only difference is the Netlify Environment variables specific to each instance. Our backend API knows how to differentiate these sites with no problem.

The problem comes, as we add more services (websites) the build times increase as well. While it used to take a few seconds for a change to become live, now the build times have gone up to as 10 minutes (including wait time while other websites are building).

Is there a way to parallelize the build process to that it takes a shorter time?

Hey @kudapara :wave:t2:

Welcome to the Community :slight_smile:

We have managed to do that with freshinabox.co.zw, freshmarket.co.zw, bottlestore.co.zw, freshfarm.co.zw and some others

Awesome! Are you using multiple Netlify Sites attached to the same Git Repository but using slightly different ENV vars and/or build commands and/or different publish directories? Sounds like it, just want to make sure I’m following your architecture correctly!

The problem comes, as we add more services (websites) the build times increase as well. While it used to take a few seconds for a change to become live, now the build times have gone up to as 10 minutes (including wait time while other websites are building).

Is there a way to parallelize the build process to that it takes a shorter time?

Hm. Well I think the answers to the first questions I asked will help me understand your issues here, but for starters a Netlify Build is really just a CLI command running in a container, so theoretically there’s nothing stopping you from finding / using a Node / Ruby / etc (whatever language your build runs in) package to help parallelize things? Would need to know a lot more about your stack to help more there…

But on another front, if you’re talking about multiple Netlify Sites building at the same time and waiting for all of them to finish, Netlify does offer concurrent builds (separate sites building at the same time) but that’s one of the primary value-add benefits of different Netlify Tiers. I’d recommend reviewing Netlify’s Pricing to see if that’s the avenue you want to go down.

Hope all of that helps for starters!


Jon

1 Like