Setting response headers only on documents

How can you set headers on all pages, but not requests? E.g. for content-security-policy, report-to etc. which should not be set on all objects.

How does one apply headers to all pages, not all requests? I’ve tried for="/" and for="*/" in netlify.toml but the former only works on the root of the site and the latter applies to all requests.

You could use for=/*.html kind of patterns, but if you serve documents at paths not ending in .html, there may not be a way to accomplish this.

Could you give us a few examples of things you are trying to cover and not to cover, if that doesn’t answer your question?

Ah then that’s a problem. It leaves the only option as setting a default header ruleset and overriding it for all known non-page extensions :confused:
This is what I’m trying to achieve: https://simonhearne.com/2019/http-headers-fast-and-secure/#configuring-headers-on-netlify
Security headers should only be sent on documents. Also it’s fair to assume that you’ll want to be able to set different headers for documents vs. assets in many cases: caching, cookies, x-frame-options etc. etc.

It doesn’t seem possible to clear headers that have been set in a previous match. I’ve tried not setting them at all, setting to empty string, setting to a single space character. All maintain the previous value.
It is not possible to set a negative match in the for="" block to exclude static assets.
I can’t find a way to test headers using netlify dev either, making this a tedious process!

e.g. with this netlify.toml we will set the correct cache-control header on static assets (i.e. those with a file extension), but they will still have the x-document-only header and value:

    [[headers]]
      # Set the default header to the one we want for documents
      for = "/*"
        [headers.values]
        cache-control = "public,max-age=3600"
        X-Document-Only = "Should not see this on assets"

    [[headers]]
      # Set the default header to the one we want for documents
      for = "/*.html"
        [headers.values]
        cache-control = "public,max-age=3600"
        X-Document-Only = "Should not see this on assets"

    [[headers]]
      # Override values for assets with periods in the filename (i.e. have an extension)
      for = "/*.*"
        [headers.values]
        cache-control = "public,max-age=360000"
        X-Document-Only = ""

Hello,

Our header rules does not have a ‘except’ feature. As such, any custom headers that matches for a path combines instead of cancelling out. That said, in your example, the first and the last rule will still apply to *.html files. You’ll need to be more explicit with each of your rules.

Yes, so how can we set rules which only apply to paths that end with ‘/’, i.e. index.html files delivered as the root of the path?
To simplify the question, what headers config will allow me to set a header value on all documents (paths ending with ‘/’) and not on any other asset type (’.js’, ‘.css’, ‘.jpg’ etc.)?

I’ve tried this as a test case:

[[headers]]
  for = "/*"
    [headers.values]
    X-Document-Only-1 = "true"
    X-Document-Only-2 = "true"
    X-Document-Only-3 = "true"
    X-Document-Only-4 = "true"

[[headers]]
  for = "/*.*"
    [headers.values]
    X-Document-Only-2 = "false"
    X-Document-Only-3 = false
    X-Document-Only-4 = " "

This results in the following on documents, as expected:

x-document-only-1: true
x-document-only-2: true
x-document-only-3: true
x-document-only-4: true

And these are the headers set on static objects, e.g. foobar.jpg:

x-document-only-1: true
x-document-only-2: false
x-document-only-3: true
x-document-only-4: true

This implies that it is not possible to set a header only on documents.

You could try for = "/*.html". This should make any headers defined for that only affect files with that extension and I think it should work for implicit files too (paths that end with a slash that has an index.html file). Could you give that a try?

Unfortunately that does not work (see https://app.netlify.com/sites/simonhearne/deploys/5e00e2e86a200d0007b165cf). Even if it did, there is still a ‘default headers’ issue here - can I set a for="*" header ruleset and override for specific paths / filetypes?

It does work. If you go to: https://5e00e2e86a200d0007b165cf--simonhearne.netlify.com/index.html, where the path includes the file name and extension, you’ll see the headers you defined. Note that header rules are cumulative so if you set the same header with different values than the ones our system sets, your values should override the default.

With regards to specific paths/ filetypes, what you see is what you get. The redirect rule I showed you is meant to make a rule apply only to the specific filetype. That said, I’m not sure what you mean for="*" but for that rule to only apply to certain paths/ filetypes. The for property is what you use to define what the rule applies to.

Perhaps I’m not understanding what your need is. If you can provide more specific details, like what you end goal is, I’ll try to provide better advice.

Hi,

I have one use case: adding Content Security Policy (CSP) headers to HTML pages.
As described here, adding such headers to non-HTML resources is contributing to header bloat: https://webhint.io/docs/user-guide/hints/hint-no-html-only-headers/

Ideally, I’d like the following URLs to have the CSP headers:

  • https://example.com/
  • https://example.com/about/
  • https://example.com/url-returning-a-404

And the following URLs to not have the CSP headers:

  • https://example.com/robots.txt
  • https://example.com/favicon.ico
  • https://example.com/assets/main.css
  • https://example.com/assets/main.js

IMHO, the cleanest way to do this would not be go via overrides, but to have an additional way to apply headers only to specific media types. It could be “additive”, allowing to apply headers only for a specific path and for a specific media type:

[[headers]]
  for = "/*"
  forMediaType = "text/html"
  [headers.values]
    Content-Security-Policy = "..."

Cheers,
Pablo

Yes, but my pages (like many folks’) are not served with a ‘.html’ extension, try https://5e00e2e86a200d0007b165cf--simonhearne.netlify.com/ instead of https://5e00e2e86a200d0007b165cf--simonhearne.netlify.com/index.html

So what I think you are saying is that it is impossible to set different headers for documents served at directory roots.

Yes this makes sense. I’m sure there are other use cases for this as well.

Hi, I filed the enhancement request referencing this post. We’ll update here if and when that request gets implemented. Thanks!

1 Like

Like @pablot I’m also after a way to target html pages so I can apply CSP and feature policy headers only to them. There’s no need for other files to have these headers (and probably many others) and they can be rather bulky. A media type matcher would be perfect for this.

Hi, @qubyte, and welcome to the Netlify community site. :smiley:

I added this as a +1 to the feature request and we’ll post an update here if/when this becomes possible.

1 Like

I’m watching too, because some headers on resource cause issues (if the issue is cached and served by a Service Worker, a “preload” HTTP Link Header can result in recursive fetch).

1 Like

I need to be able to do this as well. Webhint complain that CSP headers are being on unnecessary assets like CSS files, images etc, when all I need is for them to be on documents.

If you have a CDN in front of your Netlify deployment, you might be able to strip headers from static asset requests. E.g. here is a CloudFlare Worker which does this on simonhearne.com: https://gist.github.com/simonhearne/2715250eaccb911ebfff9e2315b2ffee