Functions CDN too slow to update to new commit

I have updated my functions with a new commit. If I call the function endpoint from a US proxy then the function is fine and updated. But if I call it without a proxy using my home IP (from Brazil), then I get an old version of function.

I understand sometimes CDNs can take some time to propagate all over the globe. But it has been over 4 hours since I last committed those changes and my functions were still not updated!

Is Netlify CDN always this slow? Is there something I can do from my end to force the CDNs to update faster?

Hmm, we intend to invalidate the cache immediately at deploy time. It takes about 2 seconds on average to purge the cache, so there’s likely either a bug or a misconfiguration.

  • potential misconfiguration: Functions are proxied internally - does your function set a Cache-Control header? If so, we will respect it and cache the answer for however long you set it. That cache is PER CDN NODE so you could see inconsistent results based on what happened on the node you’re talking to (which could change constantly) in the past
  • potential bug: we may be caching the wrong content. I don’t expect this to magically go away without our help. What’s the function path so we can do some testing? Always useful when you submit a problem with stale content is also the value of the HTTP response header called x-nf-request-id, so we can zero in on your precise problematic request in our logs and we may be able to tell what happened even in the past.

Thanks for your help in troubleshooting!

It’s just the Hello World example. It has no additional headers or anything special. The most straight-forward of the problematic ones is this:

https://gengarbobo.ca/.netlify/functions/testfunc3

When I call that URL from an US proxy, I get the correct response from the latest commit “Hello, World 4”

But when I call it from my home IP on Brazil with no proxies, then I get an old response from several hours and commits ago: “Hello, World”

Can you provide a link to the exact deploys that is showing this issue? I checked your site and I’m seeing the following issue: 2:47:03 PM: Skipping functions preparation step: /opt/build/repo/lambda not found
Which means our buildbot is not finding your functions and deploying the newer versions.

I’m sorry, I already gave up on Netlify and moved to Firebase because of this problem. So the link I provided before shouldn’t be working and the latest commit won’t work on Netlify because I already removed all the Netlify functions from the repo.

If you’re still trying to debug what went wrong. The last commit I used in Netlify before moving to Firebase was this one:
https://app.netlify.com/sites/gengarbobo/deploys/5d6fe06fe6c67300097af64a

The log says the functions built fine and should be live.

The endpoint I was having problems with can be accessed here:
https://gengarbobo.netlify.com/.netlify/functions/testfunc3

But as of today (September 6th, 2 days after I made the post) it looks like the endpoint has finally updated. I just called it again and I’m finally getting the correct response from that last commit. It’s a shame that Netlify took longer to update my functions than it took me to look up for alternatives, learn firebase from scratch, move everything there and get what I was trying to do on Netlify working perfectly on Firebase ^^

I think we found the source of the problem today. Our proxy is how we connect functions running in AWS to your namespace; this is the same functionality that is described in our docs here: https://www.netlify.com/docs/redirects/#proxying

Today we found a good way to reproduce the problem for another customer, and determined that it was a bug in that proxying configuration that essentially cached the pointer to which AWS function to run (when you deploy a new version, the old one is not removed), which we are working on a fix for.

Sorry we didn’t figure that out soon enough for you to use our service, but thank you very much for the report and we’ll follow up here when we fix the bug.