I have a static website of around 80000 pages. I know that’s a lot. My build is succeeding, but uploading the distribution fails. The first time I deployed, it failed on me after some time. See the screenshot below:
12:44:04 PM: Build script success 12:44:04 PM: Starting to deploy site from 'dist' 12:44:08 PM: Creating deploy tree asynchronously 12:44:11 PM: Creating deploy upload records 12:44:56 PM: 76173 new files to upload 12:44:56 PM: 0 new functions to upload 1:10:09 PM: Failed to upload file: posts/47480/index.html 1:10:10 PM: Failed to upload file: posts/47477/index.html 1:10:12 PM: Failed to upload file: posts/47479/index.html 1:10:17 PM: Failed to upload file: posts/47475/index.html 1:10:18 PM: Failed to upload file: posts/47476/index.html 1:10:19 PM: Failed to upload file: posts/47478/index.html 1:10:22 PM: Failed to upload file: posts/47474/index.html 1:10:51 PM: Failed to upload file: posts/47473/index.html 1:11:02 PM: Failed to upload file: posts/47481/index.html 1:11:12 PM: failed during stage 'deploying site': Failed to execute deploy: Upload cancelled: posts/47482/index.html 1:11:12 PM: Failed to upload file: posts/47482/index.html 1:11:12 PM: Failing build: Failed to deploy site
After that I manually triggered the build again and it succeeded. It looks like most of the uploading was done in the previous build and the rest was done in the second build. See the image below:
8:45:10 PM: Build script success 8:45:10 PM: Starting to deploy site from 'dist' 8:45:14 PM: Creating deploy tree asynchronously 8:45:17 PM: Creating deploy upload records 8:46:06 PM: 13704 new files to upload 8:46:06 PM: 0 new functions to upload 8:50:35 PM: Finished processing build request in 11m59.689325838s 8:54:10 PM: Starting post processing 11:47:49 PM: Post processing done 11:47:49 PM: Site is live
My question is: What can I do to have more success in my upload when building the complete website?
I know there are related issues and also some documentation around this. E.g.:
- [Common Issue] Making the most of Netlify's CDN cache
All of these don’t really give me any option, but build a less large website.
I did a lot of stuff to speed up the build already:
- I created this intra-build cache that I use to speed up the build.
- I make heavily use of webhooks to just update pages that have changed.
But sometimes I just have to build the whole site.
Help would be much appreciated!