Caching headers for prefetched assets

[wobedi]

Hello friends,

FIrst of all, love Netlify! So simple and powerful.

You can find a live deploy of my app here for reproducing the problem described below:

EMDR Online | Controllable Bilateral Stimulation

Now, my problem: I am using Webpack to bundle all my JS. On bundling, Webpack adds a content hash to the filename (‘cache busting through fingerprinting’) and adds the content hash to the JS script src in the HTML. Thus, I never have to worry about clients using outdated JS from their cache if my JS changed.

Because of this, I would like to change the caching headers of my JS files to a high max-age. Clients should cache the JS files for a very long time because my content hash fingerprinting will ensure that they get any updated JS anyways.

I was able to get that to work with Netlify in one of two ways:

1. Use the netlify.toml file to specify the custom headers
2. Enable the “Bundle JS” option in the post-processing. Your docs are not explicit that this option actually affects the caching policy but I inferred it from a blog post - might make sense to slightly tweak your docs there.

I prefer option 1 because it leaves the bundling within my control.

So what is the problem?

I also prefetch two key JS files (therapist.js and session.js). These prefetched JS files then get prefetched with the following cache-control header:

public, max-age=0, must-revalidate

This is not what I want. What I want is for these prefetched files is more something like this:

public, max-age=31556926

This second cache-control header is what I automatically get when I choose option 2 above (enable the “Bundle JS” postprocessing option). However, it only works for regular requests for this JS from the HTML script tag. It doesn’t work when I request the same JS files via prefetching.

Any ideas? What am I doing wrong?

Thanks,
Yanick

[fool]

You can modify headers for any file that isn’t asset-optimized (since those have the special handling, but also potentially create CORS problems since they are served from a different domain). You do this using our custom headers functionality:

Custom headers

For those two, you might use this in your separate headers file, or your toml file though that configuration being more verbose I don’t include it here:

/path/to/therapist.js:
   cache-control: public, max-age=31556926
/path/to/session.js
   cache-control: public, max-age=31556926

If the files have a hash in their names, you might be able to do something like put them all in a single directory, and use this syntax instead to cover all names past and present:

/path/to/*.js:
     cache-control: public, max-age=31556926

Note that both messing with these headers and cachebusting are generally regarded as antipatterns on Netlify - you should let our default caching do the work and NOT cachebust anything, since it slows your site down in many situations. This article has more details if you are curious:

[Support Guide] Making the most of Netlify's CDN cache

Last reviewed by Netlify Support - August, 2023 Our CDN derives much of its efficiency from files whose checksums don’t change between deploys. For instance, in a blog, you usually write or update one article in one file, while the rest of the content remains unchanged. In this typical use case, we don’t need to re-upload and re-process these files, since our CDN already has them stored - no checksums have changed except the one file. This also means that your visitors’ browsers don’t need to …

We won’t stop you from doing it your way, so you’re welcome to keep using it, but it creates other problems like this one:

[Support Guide] Why do I see “Uncaught SyntaxError: Unexpected token <” errors? How can I use chunking or versioning for my assets at Netlify?

Last reviewed by Netlify Support on December 2022 The error Uncaught SyntaxError: Unexpected token < is most commonly caused by your site’s code referring to an asset that is no longer available. Most commonly, this is due to a filename change in assets generated during your build. The same situation happens if you use versioned or chunked javascript where the filename changes with every deploy. Let’s look at an example: A file that was named /assets/main-123abc.js would change to /assets…

[Timothy]

@wobedi

public, max-age=31556926

I have set this on all my assets like so

for = "/assets/**/*"
[headers.values]
Cache-Control = "public, max-age=31536000

which contains my js, css, fonts and images.
But when I change a little bit of code in my javascript it won’t update the file on the website because it won’t check for the new javascript version. But why? I was under the impression that when I change the content of any of these files the etag would take care of this. Even after a full day of researching this I still cannot update. Any ideas since you posted the question ?

[hrishikesh]

@Timothy

for = "/assets/**/*"

By the way, this is redundant syntax. Netlify doesn’t use glob pattern, so /assets/* is same as what you have.

Anyways, about the actual issue, do you have a site that we can check?

[Timothy]

Ah I see, well as long as it works recursively I guess I shouldn’t worry about it.

Regarding the actual issue, I have learned a bit more about caching and the technique you guys use is actually pretty neat.

Basically my understanding is that you guys cache most (if not all) of the code and assets the site uses and return a 304 when no asset or javascript etc, changed. And that this really isn’t much of a problem because the request goes out to the nearest proxy server right?

[hrishikesh]

Yes, that’s more or less correct.