Cache not clearing on specific files

[madebyshape]

We’ve added a robots.txt file to our repo, and deployed it to the Netlify server.

We did this because noticed that Netlify was grabbing the robots.txt file from our CMS (Which is hosted on a subdomain) because it couldn’t find this file on Netlify.

But since adding this file, the file doesn’t update, and instead still shows the contents of the one from our CMS subdomain.

How long will this take to expire/clear?

Weirdly, I updated other files on the server and they updated instantly, but this robots.txt file doesn’t. Why would that be?

I can see on one of the deploy previews that it’s correct also. So i’m a bit confused.

[perry]

hmm, is there a possibility you have a .gitignore in place that is excluding that file? If yes, then that would explain why it isn’t getting updated.

[madebyshape]

We’re only deploying flat HTML files to it (After it’s been cached elsewhere) so theres nothing like that in the repo.

But the good news is that it’s fixed itself now.

Is there a cache delay on Netlify that I should be aware of? Sometimes files update instantly, but sometimes they take hours.

[perry]

well, I’m glad its fixed!

As far as delays go - there isn’t really a way to pinpoint exactly what happened here without something like an x-nf-request-id (more below), because we serve so much traffic that even knowing an approximate time window when the issue occurred barely narrows it down.

It really depends on exactly how your site is set up, also. You can learn a little more about how best to take advantage of caching here:

[Support Guide] Making the most of Netlify's CDN cache

Last reviewed by Netlify Support - August, 2023 Our CDN derives much of its efficiency from files whose checksums don’t change between deploys. For instance, in a blog, you usually write or update one article in one file, while the rest of the content remains unchanged. In this typical use case, we don’t need to re-upload and re-process these files, since our CDN already has them stored - no checksums have changed except the one file. This also means that your visitors’ browsers don’t need to …

[Support Guide] How does Netlify’s CDN handle caching files?

Last reviewed by Netlify Support on August 2023 While the topic of caching effectively is complex, Netlify’s solution is designed to be automatic and worry-free. It shouldn’t require any special configuration or build tools like cache-busting or service workers to deliver an optimal experience. What can go wrong? Some cache-busting techniques can make your site build and load more slowly. Scarier still, modifying Netlify’s default caching settings can break atomic rollbacks and deploys, which…

[Support Guide] Netlify Support asked for the 'x-nf-request-id' header? What is it and how do I find it?

Last reviewed by Netlify Support staff August 2023 What is the x-nf-request-id header? Web servers and web browsers communicate using a protocol called HTTP - which stands for “HyperText Transfer Protocol”. Both web browsers and web servers use a feature called headers as part of this protocol. The headers that web browsers send are called “request headers” because the browser in making an “HTTP request”. The web servers headers are called “response headers” because web servers send an “HTTP …