Cache issue on Netlify?

Hi @gilesbutler!

This problem mostly impacts sites that make heavy use of code-splitting in their frontend and have visitors staying a long time. We even ran into it on app.netlify.com which is also running on Netlify.
I think why it happens is pretty clear from the thread.

There are a few ways this problem can be solved. These can be combined.

Enforce the usage of the currently loaded deploy when people load the page

If you want to make sure that on loading the entrypoint html site for your app, people will only access the files of a single specific deploy, it is best to use the unique deploy url.

It might look like this: https://5e85abb1a640d40006504f77--cms-demo.netlify.com/
You get them by clicking on the timestamp on a finished deploy in the deploy overview.

This url will always point to the same set of files - remember: Netlify deploys are atomic.
If you now start to use this URL to reference any Javascript entrypoint for your app, you should not experience files disappearing while people still have your app open.

You can automate this using the DEPLOY_URL environment variable in your build.
Depending on your build setup you should be able to inject a value from an environment variable.
Your final script tag might look like this after inserting the env var:

<script src="https://5e85abb1a640d40006504f77--cms-demo.netlify.com/js/main.js></script>

Prompt visitors to reload the page if your app was updated

Many larger apps sometimes give you a notification that you should reload the page to get a new version of an app. You could implement the same for your app.

The key to this is service workers. Browsers can check for updates in service workers and notify your client-side javascript if there is a newer version available. There is a great article about the service worker lifecycle.
You should be able find resources about how to enable a service worker for your app. If you only want to use the service worker for the notification and not for offline caching, you might need to update the settings to disable local caching.

2 Likes

Hey @marcus, that’s super helpful thank you.

I’ll take a look at both of those options and figure out the best path forward.

If I come across anything useful I’ll post it back here for others in the future.

Thanks

1 Like

This issue is not solved. I am opening a new thread here to discuss my own specific issue with caching, URL rewrites and deploying via Netlify CLI:

https://answers.netlify.com/t/index-html-being-cached-and-served-instead-of-js-css-files/30139

Hi, @netlfynoob.

This might be jumping to conclusions as we have been debugging on the other topic and already found issues there that could explain this. We will keep debugging in the other topic and we can post an update here if there is actually a cache issue.

Netlify has definitely a cache system issue!

If I dont clear my browser cache, I see some missing images on my site after a rebuild.

I dont want to force my users to clear their browser cache after each new build.

Hi, @herve76. About this:

Netlify has definitely a cache system issue!

I hear this a lot and in the majority of cases the person saying this is wrong. It is true that sometimes there are issues which are Netlify’s fault but this is the exception and not the rule.

However, just like with the person above, I’m happy to troubleshoot with you to prove what the root cause is. In all too many cases though, the issue is with the way the site is designed and has nothing at all to do with Netlify.

Looking at your site I see two things that jump out.

  1. It uses a service worker.
  2. The image URLs contain hashes. An example path being:
    • /assets/static/switzerland.b598514.da314c35ddfbe8424e882a7f349d6a1f.svg

Now, whether or not those are the cause or not isn’t proven yet. Would you be willing to make a HAR recording of the issue and post it here?

If so, it will contain the information required to prove what the cause is.

You can post that information publicly or you can private message (PM) that to one of our support staff. I’ve confirmed that PMs are enabled for your forum login. Please keep in mind that only one person can see the PM and this will likely mean a slower reply than posting the information publicly. Please feel free to reply to however you prefer though.

Hi Luke! I am facing the same issue! I use netlify cli to deploy my site. I have added --prod flag for production release. But cdn cache gives 404.

This seems to be a bug to me. Please help me clear the cdn cache…
x-nf-request-id: 01FX3J02T0D43D8GTKVZG5ZAE5

I found a solution. The issue was with “Asset optimization” in build and deploy settings.


Everything works well as expected if the asset optimization is disabled.

1 Like

Thanks so much for coming back and letting us know! Glad everything is working for you now.

Hi there, feels like I have a cache problem. I’ve updated the build and it was successfully deployed and published, the deploy preview shows correct website version, but the real website has the previous version still. I tried to use “clear cache and deploy site” feature, but the result was the same.

Do you have any suggestions?

Hi, @Anton_Kuropiatnik. In order to troubleshoot we need to know how to find the incorrectly cached response.

In order to troubleshoot this, we need to be able to track the HTTP response with this issue. The fastest way to do this is to send us the x-nf-request-id header which we send with every HTTP response.

There more information about this header here:

If that header isn’t available for any reason, please send the information it replaces (or as many of these details as possible). Those details are:

  • the complete URL requested
  • the IP address for the system making the request
  • the IP address for the CDN node that responded
  • the day and time of the request (with the timezone the time is in)

If you send us that information we can research this to find out what happened.

Luke, thank you a lot for the answer and instruction, wanted to calrify:
When I go to active published deploy page, there are two buttons which help to see the build preview


The first button is in the upper part of the screen and leads to woodenshark.com, the second button is near the deploy log and leads to https://6241fe19f00675008c2a3937--woodenshark.netlify.app/

Both of them are leading to different URLs - the target URL woodenshark.com(old version) and internal netlify url(correct updated version). I use cloudflare as a hosting option and purged its cache, but it had no effect.
Woodenshark.com doesn’t have any x-nf-request-id response and the “last-modified” header refers to 2021 year, the x-nf-request-id of internal netlify url is: 01FZAW0WVPFDTRWXZP1HNGQX86

So it looks like the deploy wasn’t published though the netlify indicates that it was. I’ve cleared the browser cache, the cloudflare cache and it had no effect.

Hope it will help to locate the problem.

Sorry for the mess, I’ve found the problem in the DNS list and now everything works correctly

1 Like

We are experiencing the same issue here as well. We are deploying from CLI and using the --alias flag to set different subdomains based on the branch. Actually this issue appeared a few days ago in our case, the dev-- subdomain never updates, however the random preview URL is updated fine. Here’s the id’s from the last deploy:

  • x-nf-request-id for dev--: 01FZG2BKFGT774W9DCMYDNQVDK
  • x-nf-request-id for random preview url: 01FZG2DDJS0EM0CR8JE24DXNQX

Also, any other additional alias subdomain updates normally, the only branch affected is dev. Furthermore, if I download the zip file it is updated as well.

Hi, @Anton_Kuropiatnik. Thank you for the follow-up and for confirming the issue was a local DNS issue. I would have asked you to check this if you had not already found it based on this:

Woodenshark.com doesn’t have any x-nf-request-id response and the “last-modified” header refers to 2021 year

The HTTP responses from Netlify will always have the x-nf-request-id HTTP response header. If you were getting a response without that header - that is the clue that is was not Netlify handling the response. The most common reason for that would be local DNS issues (which you already found and have fixed).

For you @webamboos, there is something different happening. The issue here is that using --alias does not make a branch deploy. There is an open feature request for --alias deploys to be treated as “real” branch deploys here (which is a public repo):

https://github.com/netlify/cli/issues/948

However, at this time, --alias do not work as branches. The --alias option makes a one-time alias musch like the deploy id URL. This is an example of the deploy id URL from above: https://6241fe19f00675008c2a3937--woodenshark.netlify.app/.

The alias works like the 6241fe19f00675008c2a3937 deploy id in the URL. It is a one-time pointer to a specific deploy and not a branch subdomain.

The URL you are using with dev-- is locked to the last deploy of that branch using the build & deploy at Netlify. Using --alias will never update that subdomain because --alias doesn’t make branch deploys.

@Scott Hello! Is there a way to set Netlify to clear cache this way by default with every deploy with continuous integration? Thanks!

Hi, @Tim1. When you say “to clear cache this way by default” is isn’t clear which cache you are referring to.

To explain why this is unclear, these are the main reasons:

  • This is a very long thread.
  • @Scott hasn’t posted here since 2020.
  • There are two different types of caches (the build cache and the cache on the Standard Edge Network which are completely unrelated).

Would you please clarify which cache you are referring to?

If it is the Standard Edge Network cache, that is already automatic. If you are talking about the build cache, there is no way to do this currently and there probably never will be. If there are issues with the build cache, the solution is to fix this those issues.

Always building without a build cache just hides the issue by no longer using the cache. However, the build without a cache then downloads resources redundantly wasting time and data transfer resources.

This is why we prefer to resolve the issue with the build cache instead of not using the cache. The cache is an important part of preventing waste. This is way getting the cache working is always preferred to not using it at all.

​Please let us know if there are any other questions (and, if so, which cache type you are referring to).

Hello!
Thanks for the reply.
My problem is that I’m using Gatsby.js on front-end, and after I make changes in CMS, netlify runs it’s build.
But, the changes of data are often cached and I need to run another build with cache clean to see the changes on the site.
So that’s why I was wondering if there’s any way to clear cache by default every deploy.
image.png

чт, 18 авг. 2022 г. в 10:41, Luke Lawson via Netlify Support Forums <notifications@netlify.discoursemail.com>:

Hey @Tim1,

As @luke mentioned here:

…it’s currently not possible.

However, there are some workarounds that you can use meanwhile. If it’s the Gatsby cache causing issues, you could try changing your build command to gatsby clean && gatsby build, so Gatsby will get rid of its cache before building.

If that’s not an acceptable solution, another way would be to use this API endpoint: Netlify API documentation with clear_cache: true as the body to trigger a build without cache.