500 Internal Server Error on "CDN Diffing Files"

Hey @luke Luke, the file .netlify/functions/graphql.zip resulting from the local build has a size of 41.3 MB. Quite big, but within the limits I guess. Also when I deploy the same code from Github the Netlify build (and function deployment) works without a problem.

regards

Hi, @toxsick. There is a 26 second timeout and, if the upload takes longer than that, it will fail with a 500 response.

We are looking into extending that timeout and I do believe this is the root cause. Can you time the upload of that function to see it is taking longer than 26 seconds?

Hey @luke, it definately takes longer than 28 secs:

» time netlify deploy                                                                                                         ~/Coding/fuf/redwood-eval  
Deploy path:        /Users/<username>/Coding/redwood-eval/web/dist
Functions path:     /Users/<username>/Coding/redwood-eval/api/dist/functions
Configuration path: /Users/<username>/Coding/redwood-eval/netlify.toml
Deploying to draft URL...
✔ Finished hashing 13 files and 1 functions
✔ CDN requesting 5 files and 1 functions
🙉  (5/6) Uploading graphql... ›   Warning: JSONHTTPError: undefined 500
 ›   Warning: 
 ›   {
 ›      "name": "JSONHTTPError",
 ›      "status": 500,
 ›      "json": {
 ›        "status": 500,
 ›        "error": "Internal Server Error"
 ›      }
 ›   }
 ›
JSONHTTPError: Internal Server Error
    at parseResponse (~/.nvm/versions/node/v14.15.0/lib/node_modules/netlify-cli/node_modules/netlify/src/methods/response.js:12:11)
    at runMicrotasks (<anonymous>)
    at async callMethod (~/.nvm/versions/node/v14.15.0/lib/node_modules/netlify-cli/node_modules/netlify/src/methods/index.js:36:26)
netlify deploy  33.10s user 6.61s system 14% cpu 4:35.52 total

So around 4 and a half minutes before it failes.

regards

That deploy time seems quite unexpected! To confirm, you’re saying that this step:

✔ CDN requesting 5 files and 1 functions
🙉  (5/6) Uploading graphql... ›   Warning: JSONHTTPError: undefined 500
 ›   Warning: 
[...]

takes 4.5 minutes on its own? Seems like we are uploading very little so I am pretty confused by that. Or maybe you’re saying the build takes a long time (since that time is included from your time ... command - but without timestamps, I can’t tell which part is fast and which slow).

If the function upload part takes >30s that could definitely be a problem, just wanted to confirm that before adding you to the open report our developers are looking into :slight_smile:

Hey @fool, just gave it a fresh try with the latest cli version and YES it spends only 20s hashing and then around 4m uploading the graphql function before it failes:

» netlify -v
netlify-cli/2.69.0 darwin-x64 node-v14.15.0

» time netlify deploy
Deploy path:        /Users/username/Coding/fuf/redwood-eval/web/dist
Functions path:     /Users/username/Coding/fuf/redwood-eval/api/dist/functions
Configuration path: /Users/username/Coding/fuf/redwood-eval/netlify.toml
Deploying to draft URL...
✔ Finished hashing 13 files and 1 functions

-> hashing takes ~20s

✔ CDN requesting 0 files and 1 functions
    (0/1) Uploading graphql... ›   Warning: JSONHTTPError:  500
 ›   Warning: 
 ›   {
 ›     "name": "JSONHTTPError",
 ›     "status": 500,
 ›     "json": {
 ›       "status": 500,
 ›       "error": "Internal Server Error"
 ›     }
 ›   }
 ›
،    (0/1) Uploading graphql...    JSONHTTPError: Internal Server Error
netlify deploy  33.26s user 6.80s system 14% cpu 4:32.15 total

The graphql function it tries to upload is ~40MB in size:

» ls -lh .netlify/functions
-rw-r--r--  1 username  staff    39M Dec  1 10:30 graphql.zip

regards

Can I jump in here and ask how is the function bundled?
If you’re running netlify build and then netlify deploy (or netlify deploy --build) it will be bundled by zip it and ship it which tree shakes user code but not dependencies code.
This is a very safe way to bundle functions and works almost every time, but can lead to a big bundle.
The reason behind that behavior is that many node packages are not meant to be bundled and/or tree shaked like packages targeted for browsers.

A potential workaround for this is to use netlify-lambda to “pre-bundle” functions. netlify-lambda uses webpack under the hood and you can provide a custom webpack config to workaround specific issues.

Can you try configuring netlify-lambda and tell us if it works? If it doesn’t work can you share an example repo with the issue?

Hey @erez,

thanks for the context. Your remarks got me thinking why my relatively basic redwood app graphql function is already 40mb in size. I cleaned up a few dependencies (mainly babel and eslint stuff). This brought down the size of the graphql.zip created by netlify deploy --build to ~30mb and TADA the deployment worked :partying_face:

I also tried, as you suggested, to use netlify-lambda, but to be honest I didn’t understand / figure out which webpack config I should use.

For me the problem is resolved for now, I guess it could come back biting me when my function grows, but I will have an eye on that.

On the netlify side I think the error messages could be improved, in case you run into size limits with your functions, but I understand that my scenario is probably some kind of edge case in regard to function size and upload timings.

Thanks everybody!

2 Likes